中文 | English
SnapshotDB is the SQLite database format for persisting PyTorch memory profiling data. It supports multi-device (multi-GPU) snapshot storage and querying. Notably, it does more than simply converting raw pickle snapshot data into SQLite — it performs a complete "replay" of the raw memory snapshot data and records additional information during the replay, such as:
- The total memory pool size and total allocated block size after any event;
- The complete lifecycle of all memory blocks during the collection period (which event allocated them, which event freed them).
Example database file: snapshot.pkl.db
pt-snap analyzes standalone SQLite SnapshotDB files and explicitly selected
complete validated native/compatible datasets. Closed compatibility-v1 uses
immutable read-only transport; native and standalone policies remain distinct.
Acceptance, metadata/frame limits and measurements
separate schema, source/lifecycle differential and pending GUI evidence. If you start with a raw PyTorch memory snapshot pickle, import the snapshot with the built-in backend:
Security warning: Import trusted pickle files only. Pickle deserialization can execute arbitrary code. The loader rejects non-
builtinsglobal objects, butpt-snap importis not a sandbox.
pt-snap import snapshot.pkl
pt-snap import snapshot.pkl --jsonThe import command uses pt-snap-cli's first-party snapshot runtime to produce snapshot.pkl.db next to the input file by default, then updates project focus so subsequent commands can use it directly:
pt-snap query --listYou can also generate the same SnapshotDB format with another compatible producer, then point pt-snap focus at the resulting .db file.
Generated databases contain a pt_snap_metadata table with the source SHA-256, exact device
selection, import format version, producer version, and completion time. pt-snap import reuses an
existing target only when the full source SHA-256, import format version, and device selection match.
Package version changes alone do not invalidate the cache.
# Inspect devices, trace bounds, and import-metadata status
pt-snap overview snapshot.pkl.db
pt-snap overview snapshot.pkl.db --json
# Inspect provenance and compatibility metadata
pt-snap metadata snapshot.pkl.db
pt-snap metadata snapshot.pkl.db --json
# Bypass a matching cache
pt-snap import snapshot.pkl --forceimport --json reports db_path, device_id, reused, cache_miss_reason,
metadata, focus_state, and top-level focus_source. metadata --json keeps its existing field names (db_path,
status, reason, metadata) for compatibility.
Legacy or externally generated compatible databases without this table remain
queryable for templates that do not require a recognized callstack layout.
pt-snap detects the callstack layout from table columns (and treats
pt_snap_metadata.import_format_version as an auxiliary check, not the only
source of truth). Callstack templates use the matching SQL automatically.
Opening, focusing, and querying a database never writes the .db file and never
migrates it in place. Import cache invalidation still rebuilds a destination
when the current importer's format version does not match; that is separate from
read compatibility. pt-snap metadata reports missing metadata as unavailable,
and a later import of the same pickle rebuilds the database once before reuse.
Import replays allocator events and normalizes them into per-device
trace_entry_<device> and block_<device> tables. Focus chooses a database and
optional device; query templates then resolve those device-specific table names:
pt-snap focus snapshot.pkl.db --device 0
pt-snap query --template-use memory_peak
pt-snap query --template-use block --params '{"min_size": 1048576}'Use pt-snap capabilities --json for the full template catalog, or
pt-snap query --list and pt-snap query --template-info <name> to inspect
the supported query surface. event, callstack_analysis, and
active_memory_callstack_at_event work on both the current deduplicated
callstack schema and older inline-text databases; see
Callstack layout compatibility.
See Querying for the complete workflow.
| Table | Description | Records (example) |
|---|---|---|
dictionary |
Enum mapping dictionary | - |
callstack |
Deduplicated callstack text shared by all devices | 4,369 |
trace_entry_0 |
Device 0 event trace table | 8,094 |
block_0 |
Device 0 memory block table | - |
pt_snap_metadata |
First-party import provenance and cache metadata | 1 |
Naming convention: For multi-device scenarios, table names are suffixed with the device ID, e.g.,
trace_entry_1andblock_1for device 1. Thecallstacktable carries no device suffix because callstacks are shared across devices.
Records complete trace information for memory management events.
CREATE TABLE trace_entry_0 (
`id` INTEGER PRIMARY KEY,
`action` INTEGER,
`address` INTEGER,
`size` INTEGER,
`stream` INTEGER,
`allocated` INTEGER,
`active` INTEGER,
`reserved` INTEGER,
`callstackId` INTEGER
);| Column | Type | Constraints | Description | Example |
|---|---|---|---|---|
| id | INTEGER | PRIMARY KEY | Unique event ID; negative values indicate system-generated events | 1, -100 |
| action | INTEGER | — | Action type code (see below) | 4 |
| address | INTEGER | — | Memory address | 20697535234048 |
| size | INTEGER | — | Allocation size in bytes | 41943040 |
| stream | INTEGER | — | CUDA/CANN stream ID | 1276474240 |
| allocated | INTEGER | — | Total allocated bytes | 136426496 |
| active | INTEGER | — | Total active bytes | 136426496 |
| reserved | INTEGER | — | Memory pool reserved bytes | 155189248 |
| callstackId | INTEGER | NULL | Reference to callstack.id; join to read the text |
12 |
| Value | Name | Description |
|---|---|---|
| 0 | segment_map | Memory segment mapped (expandable segment) |
| 1 | segment_unmap | Memory segment unmapped |
| 2 | segment_alloc | Memory segment allocated |
| 3 | segment_free | Memory segment freed |
| 4 | alloc | Memory allocation event |
| 5 | free_requested | Free request |
| 6 | free_completed | Free completed |
| 7 | workspace_snapshot | NPU-specific workspace memory pool snapshot |
| Condition | Constraint |
|---|---|
id >= 0 |
Event IDs strictly follow the chronological order after snapshot collection starts, incrementing and unique |
id < 0 |
Synthetic events generated from raw pickle data to reconstruct Segments that existed at snapshot collection start; event type is always segment_map or segment_alloc |
id=1, action=4, address=20697535234048, size=41943040, stream=1276474240
allocated=136426496, active=136426496, reserved=155189248
callstackId=12 -> callstack.callstack:
/home/liuyekang/dev/projects/test/memory_leaks_demo.py:60 <module>
/home/liuyekang/dev/projects/test/memory_leaks_demo.py:34 main
/home/liuyekang/dev/projects/test/memory_leaks_demo.py:24 train
/home/liuyekang/dev/projects/test/memory_leaks_demo.py:12 train_one_step
Join callstack to resolve callstackId:
SELECT t.id, t.size, c.callstack
FROM trace_entry_0 t
LEFT JOIN callstack c ON c.id = t.callstackId
WHERE t.id = 1;Records detailed information about memory blocks and their lifecycle state.
CREATE TABLE block_0 (
`id` INTEGER PRIMARY KEY,
`address` INTEGER,
`size` INTEGER,
`requestedSize` INTEGER,
`state` INTEGER DEFAULT 99,
`allocEventId` INTEGER,
`freeEventId` INTEGER
);| Column | Type | SQL default | Description |
|---|---|---|---|
| id | INTEGER | — | Unique block ID; dynamic blocks use their allocation event ID, while preexisting blocks use a negative ID |
| address | INTEGER | — | Memory address |
| size | INTEGER | — | Actual allocated size (includes alignment overhead) |
| requestedSize | INTEGER | — | User-requested allocation size |
| state | INTEGER | 99 | State code (see below) |
| allocEventId | INTEGER | — | Associated allocation event ID; the producer writes -1 when it was not captured |
| freeEventId | INTEGER | — | Associated free-completion event ID; the producer writes -1 when it was not captured |
| Value | Name | Description |
|---|---|---|
| -1 | inactive | Inactive (freed) |
| 0 | active_pending_free | Active, pending free |
| 1 | active_allocated | Active, allocation |
| 99 | unknown | Unknown (default) |
| Condition | Constraint |
|---|---|
id >= 0 |
block.id matches allocEventId, pointing to the same-ID allocation event in trace_entry |
id < 0 |
The block was already allocated before snapshot collection began. Only its existence and initial state are known from pickle data; allocation time is unknown. The negative value itself carries no semantic meaning beyond uniqueness |
| Condition | Constraint |
|---|---|
allocEventId >= 0 |
block.id matches allocEventId, pointing to the same-ID allocation event in trace_entry |
allocEventId == -1 |
The allocation event for this block was not captured during snapshot collection (the block was already allocated before collection started) |
| Condition | Constraint |
|---|---|
freeEventId >= 0 |
Points to the corresponding free-completion event in trace_entry; it is independent of block.id |
freeEventId == -1 |
The free completion event for this block was not captured during snapshot collection (the block was not freed when collection ended) |
block.state is only meaningful when block.id is negative; otherwise it has no practical use.
block.size is the actual allocated size, while block.requestedSize is the
user-requested size. Both values come from the snapshot producer or replayed
allocator state. Their relationship depends on allocator configuration and must
not be inferred with one universal alignment formula.
id=-320, address=20697531023360, size=4194816, requestedSize=4194304
state=1, allocEventId=-1, freeEventId=-1
Stores enum value mappings used to decode fields like action and state.
CREATE TABLE dictionary (
`table` TEXT, -- Table name
`column` TEXT, -- Column name
`key` TEXT, -- Encoded integer value (as string)
`value` TEXT -- Original string value
);| Column | Type | Description |
|---|---|---|
| table | TEXT | Parent table name, e.g., trace_entry_0, block_0 |
| column | TEXT | Column name, e.g., action, state |
| key | TEXT | Encoded integer value as string, e.g., 4, 1 |
| value | TEXT | Original string value, e.g., alloc, active_allocated |
table=trace_entry_0, column=action, key=4, value=alloc
table=block_0, column=state, key=1, value=active_allocated
Snapshots repeat a small number of callstacks across a very large number of
events, so the text is stored once per distinct callstack and referenced by
trace_entry_<device>.callstackId.
CREATE TABLE callstack (
`id` INTEGER PRIMARY KEY,
`callstack` TEXT
);| Column | Type | Description |
|---|---|---|
| id | INTEGER | Callstack ID referenced by trace_entry_<device>.callstackId |
| callstack | TEXT | Callstack text, outermost frame first, one file:line name per line |
| Property | Guarantee |
|---|---|
| Uniqueness | One row per distinct ordered structured frame array; display text alone is not a lossless identity |
| Scope | Shared by every device in the database; the table has no device suffix |
| Coverage | Every event produced by import resolves to a row; callstackId is NULL only in externally generated databases |
Current standalone pt-snap import retains this v2 text layout and adds structured frame
tables (import_format_version = 3). Format 2 databases have the same text
layout but no original frame records.
Databases produced before callstack deduplication store inline callstack TEXT
on each trace_entry_<device> table and have no shared callstack table
(import_format_version = 1 when metadata is present).
frame stores each distinct original (filename, line, name) tuple once.
callstack_frame associates a stack with an ordered array of frame IDs:
CREATE TABLE frame (
id INTEGER PRIMARY KEY,
filename TEXT NOT NULL,
line INTEGER NOT NULL,
name TEXT NOT NULL
);
CREATE TABLE callstack_frame (
callstackId INTEGER NOT NULL,
position INTEGER NOT NULL,
frameId INTEGER NOT NULL
);
CREATE UNIQUE INDEX idx_callstack_frame_position
ON callstack_frame (callstackId, position);These tables are shared across devices. An event refers to the array through
trace_entry_<device>.callstackId; read it in position ASC order to reproduce
the original snapshot array. Position 0 is the innermost frame. The frame tree
reverses traversal to start with the outermost caller. Recursive occurrences
retain distinct positions even when they share a frame ID. Empty arrays have
no association rows and a manifest frameCount of zero. Structured fields preserve colons, whitespace, Unicode
and embedded newlines without parsing display strings.
Use event_frames to read an event's ordered frames and
active_memory_frame_tree_at_event to obtain parent/child occupancy.
See Frame memory breakdown.
Text-only databases remain readable by existing queries, but these new
queries require re-importing the original snapshot. Import detects the changed
format and rebuilds the cache; read-only analysis never migrates a database.
Format 3 applies only to standalone imports. Native sharded imports remain format 2
and compatible exports remain format 1. A separate version-2 deduplicated frame
extension supports dataset trees; see querying. callstack_frame_manifest(callstackId INTEGER PRIMARY KEY, frameCount INTEGER NOT NULL) records the expected length of every captured stack,
including zero. Frame fields must be text filename/name and an integer line.
event_frames returns no rows for absent, known-empty or corrupt arrays, never a
partial reconstruction. Trees keep known-empty captured stacks separate from
missing/corrupt evidence. Dropped tail/all links, noninteger positions and invalid
field types fail validation. Read-only queries never repair stored evidence.
pt-snap reads both layouts without rewriting the file:
| Layout | How to recognize it | Callstack text |
|---|---|---|
| v2 (current import) | trace_entry_<device>.callstackId plus shared callstack |
Join callstack.id |
| v1 (legacy) | trace_entry_<device>.callstack TEXT, no callstackId, no shared table |
Read the column directly |
event, callstack_analysis, and active_memory_callstack_at_event keep one
template name, parameter set, and output schema. The query engine selects v1 or
v2 SQL from the detected layout. Structured-frame queries additionally require
the format 3 tables; their frame evidence is not reconstructed from legacy text.
Detection is read-only: focus, query, reports, and the Python API all
open the database with SQLite mode=ro. Layout is not stored in
.pt-snap/focus.json. Conflicting columns, mixed devices, a damaged
callstack table, or metadata that disagrees with the physical schema leave
the layout unrecognized and fail only callstack templates, instead of guessing
or blocking focus, metadata, and other queries. There is no implicit v1→v2
migration.
pt-snap focus legacy.db
# Callstack layout: v1 (inline text)
pt-snap query --template-use event
pt-snap query --template-use callstack_analysisFirst-party imports create one metadata row for provenance and cache compatibility checks:
CREATE TABLE pt_snap_metadata (
id INTEGER PRIMARY KEY CHECK (id = 1),
metadata_schema_version INTEGER NOT NULL,
import_format_version INTEGER NOT NULL,
source_sha256 TEXT NOT NULL,
source_size INTEGER NOT NULL,
source_name TEXT NOT NULL,
requested_device INTEGER,
importer_name TEXT NOT NULL,
importer_version TEXT NOT NULL,
completed_at TEXT NOT NULL
);Legacy and externally produced compatible databases may omit this table. They
remain queryable, but pt-snap metadata reports their metadata as unavailable.
import --events-per-slice N publishes a distinct pt-snap-native-v2 manifest,
not a compatibility-v1 manifest or another standalone DB layout. Each finalized
shard uses the existing v2 tables plus pt_snap_block_reference (version 1), with
blockId INTEGER PRIMARY KEY, stream INTEGER NOT NULL, allocCallstack TEXT,
freeCallstack TEXT. These are text lifecycle sources, not structured frames or
an automatic cross-shard JOIN. pt_snap_metadata is required and identical in all
members; native manifest identity additionally versions source/device/capacity,
format/manifest/import/extension contracts, independently of CLI release version.
Real counts use positions per device, preserving original sparse IDs and native
OOM/workspace events. Negative boundaries do not consume capacity. Reader
validation includes full-member hashes and finalized identity/reference consistency
before bounded event addressing; it never repairs or migrates files. Aggregate
and lifecycle dataset queries remain unsupported. See the separate
native manifest protocol and
transaction/recovery guide.
P0 compatibility-v1 table order, inline layout and manifest semantics are unchanged.
import --format msinsight (alias compatibility-v1) publishes physical v1
trace_entry_<device> tables with the ninth column callstack TEXT, not views,
with the exact positional block/dictionary schema. It removes staging's unprefixed
callstack table and writes pt_snap_metadata.import_format_version=1, retaining
isolated pt_snap_block_reference text lifecycle sources. Native datasets keep their text interning and v2 metadata, plus the separate
version-2 deduplicated frame extension; standalone imports use format 3. Frames are never inferred from text.
Inline stack bytes scale with the sum of stack text over every event, not
unique stacks; conversion temporarily holds native and inline tables and may leave
SQLite free pages. Capacity does not bound blocks, loading RSS or total disk.
Compatible exports additionally carry ptSnap.identity, metadata, omitted devices
and a structural member inventory with empty digest values. Historical digest
fields are accepted as inventories; compatible import reuse compares member names,
not digest values, alongside source identity and required read-only structure.
It does not scan arbitrary row contents, so structurally valid content edits can
be reused. Recognized metadata does not attest database content integrity.
Queries and explicit validators perform independent admission checks; external v1
datasets still need no metadata. Native dataset content verification is unchanged.
For finalized compatibility datasets ONLY, bare validation and cached analysis
use mode=ro&immutable=1 after rejecting all live/dangling sidecars and aliases
before the first SQLite open. Closed/checkpointed original WAL bytes/inventory
remain unchanged. This assumes no active writer; native dataset non-WAL policy and
standalone mode=ro defaults are unchanged. See
compatible protocol and no-replace rules.