diff --git a/.github/AGENTS.md b/.github/AGENTS.md new file mode 100644 index 0000000..8c1fb6b --- /dev/null +++ b/.github/AGENTS.md @@ -0,0 +1,46 @@ + + + +# .github + +## Purpose +`.github` contains GitHub project automation: issue forms, pull request templates, snapshot provenance enforcement, release workflow, and CI verification definitions. + +## Key Files +| File | Description | +|------|-------------| +| None | GitHub metadata is organized in subdirectories. | + +## Subdirectories +| Directory | Purpose | +|-----------|---------| +| `ISSUE_TEMPLATE/` | Structured issue form YAML files (see `ISSUE_TEMPLATE/AGENTS.md`). | +| `PULL_REQUEST_TEMPLATE/` | Pull request description template (see `PULL_REQUEST_TEMPLATE/AGENTS.md`). | +| `scripts/` | Repository governance scripts, including snapshot provenance validation. | +| `workflows/` | GitHub Actions workflow definitions (see `workflows/AGENTS.md`). | + +## For AI Agents + +### Working In This Directory +- Treat workflow changes as CI-affecting; keep dependency installation and verification commands aligned with `pyproject.toml`. +- Keep issue and PR templates concise and consistent with repository scope. + +### Testing Requirements +- Validate YAML syntax for workflow or issue template edits. +- For workflow changes, mirror commands locally where practical before relying on CI. +- Run `pytest tests/test_governance.py` when provenance scripts, workflow wiring, or PR declarations change. + +### Common Patterns +- CI installs the package with development dependencies and runs lint/type/test checks. +- Snapshot runtime changes require one validated provenance decision from the PR body. + +## Dependencies + +### Internal +- `pyproject.toml` defines the Python versions, package dependencies, and tools invoked by workflows. +- `tests/` contains the suite run by CI. + +### External +- GitHub Actions runners and standard GitHub issue/PR template formats. + + diff --git a/.github/ISSUE_TEMPLATE/AGENTS.md b/.github/ISSUE_TEMPLATE/AGENTS.md new file mode 100644 index 0000000..e785ab3 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/AGENTS.md @@ -0,0 +1,43 @@ + + + +# ISSUE_TEMPLATE + +## Purpose +`ISSUE_TEMPLATE` contains structured GitHub issue forms for bugs, documentation requests, feature requests, questions, and template chooser configuration. + +## Key Files +| File | Description | +|------|-------------| +| `bug_report.yaml` | Form for reproducible defects and environment details. | +| `config.yaml` | GitHub issue template chooser configuration. | +| `documentation.yaml` | Form for documentation improvements or corrections. | +| `feature_request.yaml` | Form for proposed enhancements. | +| `question.yaml` | Form for usage questions. | + +## Subdirectories +| Directory | Purpose | +|-----------|---------| +| None | Issue templates are flat YAML files. | + +## For AI Agents + +### Working In This Directory +- Keep issue forms scoped to this Python CLI/MCP project. +- Preserve valid GitHub issue form YAML structure. + +### Testing Requirements +- Validate YAML syntax after edits. + +### Common Patterns +- Forms collect structured fields rather than relying on free-form Markdown only. + +## Dependencies + +### Internal +- Repository README/docs provide canonical terminology for issue prompts. + +### External +- GitHub issue form YAML schema. + + diff --git a/.github/PULL_REQUEST_TEMPLATE/AGENTS.md b/.github/PULL_REQUEST_TEMPLATE/AGENTS.md new file mode 100644 index 0000000..ae11934 --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE/AGENTS.md @@ -0,0 +1,42 @@ + + + +# PULL_REQUEST_TEMPLATE + +## Purpose +`PULL_REQUEST_TEMPLATE` contains the default pull request description template used for repository contributions. + +## Key Files +| File | Description | +|------|-------------| +| `pull_request_template.md` | PR body template for summaries, testing, and review context. | + +## Subdirectories +| Directory | Purpose | +|-----------|---------| +| None | PR template content is flat. | + +## For AI Agents + +### Working In This Directory +- Keep checklist items aligned with current verification expectations. +- Avoid adding process requirements that CI or project documentation does not support. +- Preserve the exact snapshot provenance decision/reason labels parsed by `.github/scripts/check_snapshot_provenance.py`. + +### Testing Requirements +- Review Markdown rendering after edits. +- Run `pytest tests/test_governance.py` when changing provenance declarations. + +### Common Patterns +- The template should prompt contributors for what changed and how it was tested. +- Snapshot runtime changes require exactly one `updated` or `no-update` decision; `no-update` also requires a specific reason. + +## Dependencies + +### Internal +- `.github/workflows/` defines automated checks referenced by PR guidance. + +### External +- GitHub pull request template Markdown support. + + diff --git a/.github/workflows/AGENTS.md b/.github/workflows/AGENTS.md new file mode 100644 index 0000000..32f4cc6 --- /dev/null +++ b/.github/workflows/AGENTS.md @@ -0,0 +1,46 @@ + + + +# workflows + +## Purpose +`workflows` contains GitHub Actions definitions for continuous integration and release automation. + +## Key Files +| File | Description | +|------|-------------| +| `test.yml` | PR provenance gate plus lint, type, test-matrix, coverage, build, and wheel smoke checks. | +| `release.yml` | Release workflow for building and publishing package artifacts. | + +## Subdirectories +| Directory | Purpose | +|-----------|---------| +| None | Workflow YAML files are flat. | + +## For AI Agents + +### Working In This Directory +- Keep workflow Python versions and commands aligned with `pyproject.toml`. +- Treat release workflow edits as high-impact and verify syntax carefully. +- Do not bypass tests, lint, type checks, or publishing safeguards without explicit instruction. +- Keep snapshot provenance inputs and declaration parsing aligned with `.github/scripts/check_snapshot_provenance.py` and the PR template. + +### Testing Requirements +- Validate YAML syntax. +- Run equivalent local commands when changing CI command sequences. +- Run `pytest tests/test_governance.py tests/test_release_workflow.py` for provenance or release workflow changes. + +### Common Patterns +- Workflows install the package with development extras before verification. +- The test workflow defines independent provenance, lint, and Python-version test jobs that may run concurrently. + +## Dependencies + +### Internal +- `pyproject.toml` defines package metadata, dev extras, and tool configuration. +- `tests/` and source files are exercised by CI. + +### External +- GitHub Actions hosted runners and packaging/publishing actions. + + diff --git a/AGENTS.md b/AGENTS.md index 1f93e91..972e389 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -26,6 +26,15 @@ Run these commands from the repository root. coverage reports under `test_reports/`; use direct `pytest` commands unless that local environment is intentionally available. +## Scoped Guidance + +| Scope | Guide | Responsibility | +| --- | --- | --- | +| GitHub automation | [.github/AGENTS.md](.github/AGENTS.md) | Issue/PR templates, provenance guard, CI, and release workflows | +| User documentation | [docs/AGENTS.md](docs/AGENTS.md) | English/Chinese guides, API docs, navigation, and legal evidence | +| Package source | [src/AGENTS.md](src/AGENTS.md) | Installable package boundaries and source-layout rules | +| Tests | [tests/AGENTS.md](tests/AGENTS.md) | Cross-surface contracts, service/query/runtime suites, and trusted fixtures | + ## Runtime Topology | Surface | Entry point | Responsibility | diff --git a/README.md b/README.md index e9db587..2f8103b 100644 --- a/README.md +++ b/README.md @@ -6,6 +6,8 @@ A command-line tool for analyzing PyTorch memory snapshots. Set a snapshot datab ## Installation +From a source checkout: + ```bash pip install -e . ``` @@ -25,7 +27,7 @@ pt-snap query --list ```bash # Set the snapshot database and device -pt-snap focus examples/snapshot_expandable.pkl.db --device 0 +pt-snap focus snapshot.pkl.db --device 0 # List available queries pt-snap query --list @@ -71,21 +73,12 @@ See the [full quick start guide](docs/en/quickstart.md) for a walkthrough. pt-snap-mcp ``` -The server exposes the following tools: - -| Tool | Description | -|------|-------------| -| `get_focus` | Get the current analysis focus | -| `set_focus` | Set focus to a database and optional device | -| `list_templates` | List available query templates | -| `get_template_info` | Get template details and parameters | -| `execute_query` | Run a query template against the focused database | -| `get_database_metadata` | Inspect import metadata for the focused or specified database | - See the [MCP guide](docs/en/mcp.md) for setup and usage details. ## Documentation +See the [documentation index](docs/README.md) for all English and Chinese guides. + | Topic | Guide | |-------|-------| | Getting started | [Quick Start](docs/en/quickstart.md) | @@ -94,61 +87,14 @@ See the [MCP guide](docs/en/mcp.md) for setup and usage details. | Splitting snapshots | [Splitting Snapshots](docs/en/splitting.md) | | MCP server | [MCP Guide](docs/en/mcp.md) | | Database format | [SnapshotDB Schema](docs/en/database.md) | -| Python API | [ResultMapper API](docs/en/result-mapper-api.md) | - -## Query Templates - -9 built-in templates across 3 categories: - -- **Basic**: `block`, `event`, `allocation` -- **Statistical**: `active_blocks_at_event`, `allocator_gap`, `callstack_analysis`, `memory_peak` -- **Business**: `active_memory_callstack_at_event`, `leak_detection` - -See [Querying](docs/en/querying.md) for details. - -## Project Structure - -``` -pt-snap-cli/ -├── src/ -│ └── pt_snap_cli/ -│ ├── cli.py # CLI entry point -│ ├── context.py # Database context manager -│ ├── config.py # Focus management -│ ├── api.py # Python API layer -│ ├── query/ -│ │ ├── builder.py # Query builder -│ │ ├── executor.py # Query executor -│ │ ├── mapper.py # Result mapper -│ │ ├── registry.py # Query registry -│ │ ├── condition.py # Query conditions -│ │ ├── config.py # Query configuration -│ │ └── templates/ # Query templates -│ ├── models/ # Data models -│ └── mcp/ # MCP server for agent integration -├── tests/ # Test files -├── examples/ # Example data -└── docs/ # Documentation -``` +| Python API | [SnapshotAnalyzer API](docs/en/snapshot-analyzer-api.md) | +| Result mapping utility | [ResultMapper API](docs/en/result-mapper-api.md) | ## Development ```bash +pip install -e ".[dev]" # Install development dependencies pytest # Run all tests -./tests/run_tests.sh # Full test run with coverage -black . && ruff check . # Format and lint +black --check . && ruff check . # Check formatting and lint python -m build # Build sdist and wheel ``` - -### CLI/MCP semantic contract - -The CLI and MCP server must keep using the shared API/core services as the single -semantic source of truth. The CLI may render results as terminal text and the MCP -server may return JSON-like tool payloads, but normalized meanings must stay in -sync for focus handling, query template listing, template metadata, query -execution, and shared error cases. - -`tests/test_contract_cli_mcp.py` compares normalized CLI output with MCP tool -results on the same inputs and fixtures. When adding a new shared CLI/MCP capability, -update or add a contract case in that file so CI fails if either adapter drifts -from the shared behavior. diff --git a/README_zh.md b/README_zh.md index c8134ef..0c6dd0d 100644 --- a/README_zh.md +++ b/README_zh.md @@ -6,6 +6,8 @@ ## 安装 +从源码 checkout 安装: + ```bash pip install -e . ``` @@ -21,7 +23,7 @@ pt-snap import snapshot.pkl pt-snap metadata snapshot.pkl.db # 设置快照数据库和设备 -pt-snap focus examples/snapshot_expandable.pkl.db --device 0 +pt-snap focus snapshot.pkl.db --device 0 # 列出可用查询 pt-snap query --list @@ -67,21 +69,12 @@ pt-snap split snapshot.pkl --slices 4 --output snapshot-slices pt-snap-mcp ``` -服务器暴露以下工具: - -| 工具 | 说明 | -|------|------| -| `get_focus` | 获取当前分析焦点 | -| `set_focus` | 设置焦点到指定数据库和设备 | -| `list_templates` | 列出可用的查询模板 | -| `get_template_info` | 获取模板详情和参数 | -| `execute_query` | 对焦点数据库执行查询模板 | -| `get_database_metadata` | 查看焦点或指定数据库的导入 metadata | - 详见 [MCP 指南](docs/zh/mcp.md)。 ## 文档 +全部中英文指南见[文档索引](docs/README.md)。 + | 主题 | 指南 | |------|------| | 入门指南 | [Quick Start](docs/zh/quickstart.md) | @@ -90,48 +83,14 @@ pt-snap-mcp | 拆分快照 | [拆分快照](docs/zh/splitting.md) | | MCP 服务器 | [MCP 指南](docs/zh/mcp.md) | | 数据库格式 | [SnapshotDB Schema](docs/zh/database.md) | -| Python API | [ResultMapper API](docs/zh/result-mapper-api.md) | - -## 查询模板 - -9 个内置模板,3 个分类: - -- **Basic**: `block`, `event`, `allocation` -- **Statistical**: `active_blocks_at_event`, `allocator_gap`, `callstack_analysis`, `memory_peak` -- **Business**: `active_memory_callstack_at_event`, `leak_detection` - -详见 [Querying](docs/zh/querying.md)。 - -## 项目结构 - -``` -pt-snap-cli/ -├── src/ -│ └── pt_snap_cli/ -│ ├── cli.py # CLI 入口 -│ ├── context.py # 数据库连接管理器 -│ ├── config.py # 焦点管理 -│ ├── api.py # Python API 层 -│ ├── query/ -│ │ ├── builder.py # 查询构建器 -│ │ ├── executor.py # 查询执行器 -│ │ ├── mapper.py # 结果映射器 -│ │ ├── registry.py # 查询注册表 -│ │ ├── condition.py # 查询条件 -│ │ ├── config.py # 查询配置 -│ │ └── templates/ # 查询模板 -│ ├── models/ # 数据模型 -│ └── mcp/ # MCP 服务器(Agent 集成) -├── tests/ # 测试文件 -├── examples/ # 示例数据 -└── docs/ # 文档 -``` +| Python API | [SnapshotAnalyzer API](docs/zh/snapshot-analyzer-api.md) | +| 结果映射工具 | [ResultMapper API](docs/zh/result-mapper-api.md) | ## 开发 ```bash +pip install -e ".[dev]" # 安装开发依赖 pytest # 运行所有测试 -./tests/run_tests.sh # 完整测试(含覆盖率) -black . && ruff check . # 格式化和 lint +black --check . && ruff check . # 检查格式和 lint python -m build # 构建 sdist 和 wheel ``` diff --git a/docs/AGENTS.md b/docs/AGENTS.md new file mode 100644 index 0000000..498bb61 --- /dev/null +++ b/docs/AGENTS.md @@ -0,0 +1,45 @@ + + + +# docs + +## Purpose +`docs` contains end-user and API documentation for the CLI, focus management, query templates, MCP integration, snapshot database format, high-level and result mapping Python APIs, and retained legal evidence. It is split into English and Chinese language trees with a top-level README for navigation. + +## Key Files +| File | Description | +|------|-------------| +| `README.md` | Documentation landing page and language/topic navigation. | + +## Subdirectories +| Directory | Purpose | +|-----------|---------| +| `en/` | English documentation guides (see `en/AGENTS.md`). | +| `zh/` | Chinese documentation guides (see `zh/AGENTS.md`). | +| `legal/` | Governance evidence referenced by the documentation index and snapshot provenance record. | + +## For AI Agents + +### Working In This Directory +- Keep English and Chinese docs synchronized when changing user-visible behavior. +- Verify command names, options, template names, and MCP tool names against source code before updating docs. +- Prefer updating existing guides over creating new documentation pages. + +### Testing Requirements +- For documentation-only changes, review rendered Markdown structure and verify referenced commands/options exist. +- If examples depend on behavior changes, run the matching CLI or service tests. + +### Common Patterns +- The two language trees mirror the same topics: quick start, focus management, querying, snapshot splitting, MCP, database schema, SnapshotAnalyzer API, and result mapper API. + +## Dependencies + +### Internal +- `src/pt_snap_cli/cli.py` defines documented CLI commands and options. +- `src/pt_snap_cli/mcp/server.py` defines documented MCP tools, resource, and prompt. +- `src/pt_snap_cli/query/templates/` defines documented built-in template names and parameters. + +### External +- Markdown is the only documentation format used here. + + diff --git a/docs/README.md b/docs/README.md index d7c4a52..69fba51 100644 --- a/docs/README.md +++ b/docs/README.md @@ -5,11 +5,15 @@ | Topic | File | |-------|------| | Quick Start | [en/quickstart.md](en/quickstart.md) | -| Context Management | [en/focus-management.md](en/focus-management.md) | +| Importing snapshots | [en/quickstart.md](en/quickstart.md) | +| Focus Management | [en/focus-management.md](en/focus-management.md) | | Querying | [en/querying.md](en/querying.md) | +| Reports | [en/querying.md](en/querying.md) | | Splitting Snapshots | [en/splitting.md](en/splitting.md) | | MCP server | [en/mcp.md](en/mcp.md) | | Database Schema | [en/database.md](en/database.md) | +| Import metadata | [en/database.md](en/database.md) | +| SnapshotAnalyzer API | [en/snapshot-analyzer-api.md](en/snapshot-analyzer-api.md) | | ResultMapper API | [en/result-mapper-api.md](en/result-mapper-api.md) | ## 中文 @@ -17,9 +21,17 @@ | 主题 | 文件 | |------|------| | 快速开始 | [zh/quickstart.md](zh/quickstart.md) | +| 导入快照 | [zh/quickstart.md](zh/quickstart.md) | | Focus 管理 | [zh/focus-management.md](zh/focus-management.md) | | 运行查询 | [zh/querying.md](zh/querying.md) | +| 分析报告 | [zh/querying.md](zh/querying.md) | | 拆分快照 | [zh/splitting.md](zh/splitting.md) | | MCP 服务器 | [zh/mcp.md](zh/mcp.md) | | 数据库格式 | [zh/database.md](zh/database.md) | +| 导入 metadata | [zh/database.md](zh/database.md) | +| SnapshotAnalyzer API | [zh/snapshot-analyzer-api.md](zh/snapshot-analyzer-api.md) | | ResultMapper API | [zh/result-mapper-api.md](zh/result-mapper-api.md) | + +## Governance + +- [Snapshot runtime MIT relicensing evidence](legal/mem%73napdump-mit-relicensing.md) diff --git a/docs/en/AGENTS.md b/docs/en/AGENTS.md new file mode 100644 index 0000000..a18e379 --- /dev/null +++ b/docs/en/AGENTS.md @@ -0,0 +1,47 @@ + + + +# en + +## Purpose +`docs/en` contains the English documentation set for installing and using `pt-snap-cli`, managing focus, querying and splitting snapshots, running the MCP server, understanding the SQLite schema, and using the high-level and result mapping Python APIs. + +## Key Files +| File | Description | +|------|-------------| +| `quickstart.md` | End-to-end installation and first-query walkthrough. | +| `focus-management.md` | Focus resolution, project focus files, environment variables, and global config behavior. | +| `querying.md` | Query listing, template info, parameter usage, result limits, and built-in templates. | +| `splitting.md` | Snapshot slicing strategies, output formats, replay validation, and publication guarantees. | +| `mcp.md` | MCP server setup, tools, resources, prompts, and agent usage. | +| `database.md` | Snapshot SQLite table/schema reference. | +| `snapshot-analyzer-api.md` | High-level Python API for focus, template discovery, queries, and import metadata. | +| `result-mapper-api.md` | Result mapping API documentation. | + +## Subdirectories +| Directory | Purpose | +|-----------|---------| +| None | English docs are flat topic files. | + +## For AI Agents + +### Working In This Directory +- Keep examples executable against current CLI options and template names. +- Mirror user-visible changes in `../zh/` when appropriate. + +### Testing Requirements +- Verify referenced commands and options against `src/pt_snap_cli/cli.py`. +- Verify template details against `src/pt_snap_cli/query/templates/`. + +### Common Patterns +- Topic files correspond to major product areas rather than source modules. + +## Dependencies + +### Internal +- CLI, MCP, query template, and schema details should be checked against `src/pt_snap_cli/` before editing. + +### External +- Markdown only. + + diff --git a/docs/en/database.md b/docs/en/database.md index bef6ba4..9aea644 100644 --- a/docs/en/database.md +++ b/docs/en/database.md @@ -1,6 +1,6 @@ # SnapshotDB Schema -[English](database.md) | [中文](../zh/database.md) +[中文](../zh/database.md) | English ## Overview @@ -8,7 +8,7 @@ SnapshotDB is the SQLite database format for persisting PyTorch memory profiling 1. The total memory pool size and total allocated block size after any event; 2. The complete lifecycle of all memory blocks during the collection period (which event allocated them, which event freed them). -**Example database file**: `snapshot_expandable.pkl.db` +**Example database file**: `snapshot.pkl.db` --- @@ -59,7 +59,7 @@ optional device; query templates then resolve those device-specific table names: ```bash pt-snap focus snapshot.pkl.db --device 0 pt-snap query --template-use memory_peak -pt-snap query --template-use block --params '{"state": 1}' +pt-snap query --template-use block --params '{"min_size": 1048576}' ``` Use `pt-snap query --list` and `pt-snap query --template-info ` to inspect @@ -76,6 +76,7 @@ the supported query surface. See [Querying](querying.md) for the complete workfl | `dictionary` | Enum mapping dictionary | - | | `trace_entry_0` | Device 0 event trace table | 8,094 | | `block_0` | Device 0 memory block table | - | +| `pt_snap_metadata` | First-party import provenance and cache metadata | 1 | > **Naming convention**: For multi-device scenarios, table names are suffixed with the device ID, e.g., `trace_entry_1` and `block_1` for device 1. @@ -92,13 +93,13 @@ Records complete trace information for memory management events. ```sql CREATE TABLE trace_entry_0 ( `id` INTEGER PRIMARY KEY, - `action` INTEGER NOT NULL, - `address` INTEGER NOT NULL, - `size` INTEGER NOT NULL, - `stream` INTEGER NOT NULL, - `allocated` INTEGER NOT NULL, - `active` INTEGER NOT NULL, - `reserved` INTEGER NOT NULL, + `action` INTEGER, + `address` INTEGER, + `size` INTEGER, + `stream` INTEGER, + `allocated` INTEGER, + `active` INTEGER, + `reserved` INTEGER, `callstack` TEXT ); ``` @@ -108,13 +109,13 @@ CREATE TABLE trace_entry_0 ( | Column | Type | Constraints | Description | Example | |--------|------|-------------|-------------|---------| | id | INTEGER | PRIMARY KEY | Unique event ID; negative values indicate system-generated events | `1`, `-100` | -| action | INTEGER | NOT NULL | Action type code (see below) | `4` | -| address | INTEGER | NOT NULL | Memory address | `20697535234048` | -| size | INTEGER | NOT NULL | Allocation size in bytes | `41943040` | -| stream | INTEGER | NOT NULL | CUDA/CANN stream ID | `1276474240` | -| allocated | INTEGER | NOT NULL | Total allocated bytes | `136426496` | -| active | INTEGER | NOT NULL | Total active bytes | `136426496` | -| reserved | INTEGER | NOT NULL | Memory pool reserved bytes | `155189248` | +| action | INTEGER | — | Action type code (see below) | `4` | +| address | INTEGER | — | Memory address | `20697535234048` | +| size | INTEGER | — | Allocation size in bytes | `41943040` | +| stream | INTEGER | — | CUDA/CANN stream ID | `1276474240` | +| allocated | INTEGER | — | Total allocated bytes | `136426496` | +| active | INTEGER | — | Total active bytes | `136426496` | +| reserved | INTEGER | — | Memory pool reserved bytes | `155189248` | | callstack | TEXT | NULL | Callstack information (multi-line text) | See example | #### Action Type Codes @@ -160,26 +161,26 @@ Records detailed information about memory blocks and their lifecycle state. ```sql CREATE TABLE block_0 ( `id` INTEGER PRIMARY KEY, - `address` INTEGER NOT NULL, - `size` INTEGER NOT NULL, - `requestedSize` INTEGER NOT NULL DEFAULT 99, - `state` INTEGER NOT NULL DEFAULT 99, - `allocEventId` INTEGER DEFAULT -1, - `freeEventId` INTEGER DEFAULT -1 + `address` INTEGER, + `size` INTEGER, + `requestedSize` INTEGER, + `state` INTEGER DEFAULT 99, + `allocEventId` INTEGER, + `freeEventId` INTEGER ); ``` #### Columns -| Column | Type | Default | Description | +| Column | Type | SQL default | Description | |--------|------|---------|-------------| -| id | INTEGER | — | Unique block ID, typically negative | +| id | INTEGER | — | Unique block ID; dynamic blocks use their allocation event ID, while preexisting blocks use a negative ID | | address | INTEGER | — | Memory address | | size | INTEGER | — | Actual allocated size (includes alignment overhead) | -| requestedSize | INTEGER | 99 | User-requested allocation size | +| requestedSize | INTEGER | — | User-requested allocation size | | state | INTEGER | 99 | State code (see below) | -| allocEventId | INTEGER | -1 | Associated allocation event ID | -| freeEventId | INTEGER | -1 | Associated free event ID | +| allocEventId | INTEGER | — | Associated allocation event ID; the producer writes `-1` when it was not captured | +| freeEventId | INTEGER | — | Associated free-completion event ID; the producer writes `-1` when it was not captured | #### State Codes @@ -208,7 +209,7 @@ CREATE TABLE block_0 ( | Condition | Constraint | |-----------|------------| -| `freeEventId >= 0` | `block.id` matches `freeEventId`, pointing to the same-ID free event in `trace_entry` | +| `freeEventId >= 0` | Points to the corresponding free-completion event in `trace_entry`; it is independent of `block.id` | | `freeEventId == -1` | The free completion event for this block was not captured during snapshot collection (the block was not freed when collection ended) | #### State Usage @@ -217,11 +218,10 @@ CREATE TABLE block_0 ( #### Size Calculation -`block.size` is the actual allocated size (including alignment overhead), while `block.requestedSize` is the user-requested size. The calculation is: - -``` -size = math.ceil((requestedSize + 32) / 512) * 512 -``` +`block.size` is the actual allocated size, while `block.requestedSize` is the +user-requested size. Both values come from the snapshot producer or replayed +allocator state. Their relationship depends on allocator configuration and must +not be inferred with one universal alignment formula. #### Data Example @@ -242,8 +242,8 @@ Stores enum value mappings used to decode fields like `action` and `state`. CREATE TABLE dictionary ( `table` TEXT, -- Table name `column` TEXT, -- Column name - `key` TEXT, -- Original string value - `value` TEXT -- Encoded integer value (as string) + `key` TEXT, -- Encoded integer value (as string) + `value` TEXT -- Original string value ); ``` @@ -253,12 +253,35 @@ CREATE TABLE dictionary ( |--------|------|-------------| | table | TEXT | Parent table name, e.g., `trace_entry_0`, `block_0` | | column | TEXT | Column name, e.g., `action`, `state` | -| key | TEXT | Original string value, e.g., `alloc`, `active_allocated` | -| value | TEXT | Encoded integer value as string, e.g., `4`, `1` | +| key | TEXT | Encoded integer value as string, e.g., `4`, `1` | +| value | TEXT | Original string value, e.g., `alloc`, `active_allocated` | #### Data Example ``` -table=trace_entry_0, column=action, key=alloc, value=4 -table=block_0, column=state, key=active_allocated, value=1 +table=trace_entry_0, column=action, key=4, value=alloc +table=block_0, column=state, key=1, value=active_allocated +``` + +### 4. pt_snap_metadata — Import Metadata Table + +First-party imports create one metadata row for provenance and cache +compatibility checks: + +```sql +CREATE TABLE pt_snap_metadata ( + id INTEGER PRIMARY KEY CHECK (id = 1), + metadata_schema_version INTEGER NOT NULL, + import_format_version INTEGER NOT NULL, + source_sha256 TEXT NOT NULL, + source_size INTEGER NOT NULL, + source_name TEXT NOT NULL, + requested_device INTEGER, + importer_name TEXT NOT NULL, + importer_version TEXT NOT NULL, + completed_at TEXT NOT NULL +); ``` + +Legacy and externally produced compatible databases may omit this table. They +remain queryable, but `pt-snap metadata` reports their metadata as unavailable. diff --git a/docs/en/focus-management.md b/docs/en/focus-management.md index 1ac9be2..3a36614 100644 --- a/docs/en/focus-management.md +++ b/docs/en/focus-management.md @@ -2,7 +2,10 @@ [中文](../zh/focus-management.md) | English -`pt-snap` supports project-scoped focus so different terminals, agents, or working directories can analyze different databases and devices without interfering with each other. +`pt-snap` supports project-scoped focus. Processes that resolve the same nearest +`.pt-snap/focus.json` share that focus, while different project directories can +select different databases and devices. Use a session override when terminals or +agents in the same project must remain isolated. ## Resolution Priority @@ -84,9 +87,13 @@ pt-snap config --clear # Clear global configuration | Scope | File | |-------|------| -| Project | `.pt-snap/focus.json` (git-ignored, per-project) | +| Project | `.pt-snap/focus.json` (shared by processes in the project scope) | | Global | `~/.config/pt-snap-cli/config.json` | +Project focus can contain an absolute local database path. Add `.pt-snap/` to the +project's `.gitignore` when the file should remain local; pt-snap does not modify +ignore rules automatically. + ### Focus File Format ```json diff --git a/docs/en/mcp.md b/docs/en/mcp.md index 73b5b2a..a63388d 100644 --- a/docs/en/mcp.md +++ b/docs/en/mcp.md @@ -6,6 +6,8 @@ ## Installation +From a source checkout: + ```bash pip install -e . ``` @@ -45,7 +47,7 @@ This starts a FastMCP server that exposes tools for analyzing PyTorch memory sna ## Typical Workflow -1. Call `set_focus` with `db_path` to point to a snapshot file +1. Call `set_focus` with `db_path` to point to a SnapshotDB `.db` file 2. Call `list_templates` to discover available queries 3. Call `get_template_info` with a template name to see its parameters 4. Call `execute_query` with the template name and parameters @@ -54,7 +56,7 @@ This starts a FastMCP server that exposes tools for analyzing PyTorch memory sna ## Example Usage ```python -# Set focus to a snapshot +# Set focus to a SnapshotDB file set_focus(db_path="/path/to/snapshot.db", device_id=0) # List templates @@ -63,7 +65,7 @@ list_templates() # Get template details get_template_info("leak_detection") -# Returns: {"name": "leak_detection", "parameters": [...], ...} +# Returns: {"name": "leak_detection", "parameters": {"min_size": {...}}, ...} # Run a query execute_query("leak_detection", params={"min_size": 1024}) diff --git a/docs/en/querying.md b/docs/en/querying.md index 7d1c0fe..5183bd5 100644 --- a/docs/en/querying.md +++ b/docs/en/querying.md @@ -1,26 +1,29 @@ # Querying -[English](querying.md) | [中文](../zh/querying.md) +[中文](../zh/querying.md) | English Run memory analysis queries against your snapshot database. ## The Query Command ```bash -pt-snap query [--template-use ] [--params ] [--device ] [--list] [--template-info