From faf912149192b591c7f41d64082d4e65404840c1 Mon Sep 17 00:00:00 2001 From: codeasier Date: Sat, 8 Aug 2026 00:35:16 +0800 Subject: [PATCH] docs: consolidate agent guidance --- .agents/quickstart/01-cli.md | 58 ------------- .agents/quickstart/02-context.md | 63 -------------- .agents/quickstart/03-models.md | 71 --------------- .agents/quickstart/04-query.md | 81 ------------------ .agents/quickstart/ARCHITECTURE.md | 75 ---------------- .agents/quickstart/README-for-agents.md | 88 ------------------- .gitignore | 36 +++++++- AGENTS.md | 109 ++++++++++++++++++++++++ CLAUDE.md | 1 + 9 files changed, 144 insertions(+), 438 deletions(-) delete mode 100644 .agents/quickstart/01-cli.md delete mode 100644 .agents/quickstart/02-context.md delete mode 100644 .agents/quickstart/03-models.md delete mode 100644 .agents/quickstart/04-query.md delete mode 100644 .agents/quickstart/ARCHITECTURE.md delete mode 100644 .agents/quickstart/README-for-agents.md create mode 100644 AGENTS.md create mode 100644 CLAUDE.md diff --git a/.agents/quickstart/01-cli.md b/.agents/quickstart/01-cli.md deleted file mode 100644 index f375eb9..0000000 --- a/.agents/quickstart/01-cli.md +++ /dev/null @@ -1,58 +0,0 @@ -# CLI 模块 - -## 职责 - -提供 `pt-snap` 命令行工具,支持数据库设置、查询执行等核心功能。 - -## 核心组件 - -### `app` (Typer) -Typer 应用实例,管理所有 CLI 命令。 - -### `focus_database()` -设置当前分析的数据库文件,验证有效性并显示可用设备。 - -### `query_database()` -执行查询命令,支持: -- `--template-use`: 指定查询模板 -- `--params`: JSON 格式参数 -- `--device`: 指定设备 ID -- `--list`: 列出可用模板 - -### `version_callback()` -版本信息回调函数。 - -## 依赖关系 - -- **依赖**:`Context` (context.py) - 数据库上下文 -- **依赖**:`QueryExecutor` (query/executor.py) - 查询执行 -- **依赖**:`typer` - CLI 框架 - -## 使用示例 - -```python -from pt_snap_cli.cli import app - -# 运行 CLI -app() -``` - -```bash -# 查看版本 -pt-snap --version - -# 设置数据库 -pt-snap focus examples/snapshot.pkl.db - -# 执行查询 -pt-snap query snapshot.pkl.db --template-use leak_detection --params '{"min_size": 1024}' - -# 列出模板 -pt-snap query snapshot.pkl.db --list -``` - -## 相关文件 - -- [cli.py](../../src/pt_snap_cli/cli.py) - 完整实现 -- [context.py](../../src/pt_snap_cli/context.py) - 数据库上下文 -- [query/executor.py](../../src/pt_snap_cli/query/executor.py) - 查询执行器 diff --git a/.agents/quickstart/02-context.md b/.agents/quickstart/02-context.md deleted file mode 100644 index 97671e1..0000000 --- a/.agents/quickstart/02-context.md +++ /dev/null @@ -1,63 +0,0 @@ -# Context 模块 - -## 职责 - -管理 SQLite 数据库连接,提供只读访问模式和设备发现功能。 - -## 核心组件 - -### `Context` 类 - -**属性:** -- `db_path` - 数据库文件路径 -- `device_ids` - 可用的设备 ID 列表 - -**方法:** -- `__init__(db_path, devices)` - 初始化并验证 schema -- `connect()` - 上下文管理器,以只读模式打开连接 -- `close()` - 关闭连接 -- `cursor()` - 获取数据库游标 -- `_validate_schema()` - 验证数据库 schema -- `_discover_device_ids()` - 从表名发现设备 ID - -### 异常类 - -- `DatabaseNotFoundError` - 数据库文件不存在 -- `SchemaVersionError` - Schema 无效或不兼容 - -## 依赖关系 - -- **依赖**:`sqlite3` - SQLite 连接 -- **被依赖**:`cli.py` - CLI 命令使用 -- **被依赖**:`query/executor.py` - 查询执行使用 - -## 设计特点 - -1. **只读模式**:使用 `file:...?mode=ro` URI 防止数据修改 -2. **延迟连接**:首次访问时才建立连接 -3. **设备缓存**:`device_ids` 缓存避免重复查询 -4. **Schema 验证**:启动时验证 `dictionary` 表存在 - -## 使用示例 - -```python -from pt_snap_cli.context import Context - -# 初始化并验证 -ctx = Context("snapshot.pkl.db") - -# 获取设备列表 -devices = ctx.device_ids # [0, 1, 2] - -# 使用上下文管理器 -with ctx.connect() as conn: - cursor = conn.cursor() - cursor.execute("SELECT * FROM trace_entry_0") - -# 自动关闭 -``` - -## 相关文件 - -- [context.py](../../src/pt_snap_cli/context.py) - 完整实现 -- [cli.py](../../src/pt_snap_cli/cli.py) - 使用 Context 的 CLI 命令 diff --git a/.agents/quickstart/03-models.md b/.agents/quickstart/03-models.md deleted file mode 100644 index 9ceff96..0000000 --- a/.agents/quickstart/03-models.md +++ /dev/null @@ -1,71 +0,0 @@ -# Models 模块 - -## 职责 - -定义 PyTorch 内存快照的数据模型,包括内存块和内存事件。 - -## 核心组件 - -### `MemoryBlock` 类 -内存块模型,表示一次内存分配。 - -**关键属性:** -- `address` - 内存地址 -- `size` - 分配大小(字节) -- `requested_size` - 用户请求的大小 -- `state` - 内存块状态 -- `alloc_event_id` / `free_event_id` - 生命周期事件 ID - -### `MemoryEvent` 类 -内存事件模型,表示内存分配/释放事件。 - -**关键属性:** -- `id` - 事件 ID -- `action` - 事件类型 -- `allocated` / `active` / `reserved` - 事件后的内存统计 - -### `EventType` 枚举 -事件类型定义: -- `UNKNOWN` - 未知类型 -- `ALLOC` - 分配事件 -- `FREE` - 释放事件 - -## 依赖关系 - -- **被依赖**:`query/` - 查询模块使用模型组织结果 -- **依赖**:标准库和 PyTorch 相关类型 - -## 使用示例 - -```python -from pt_snap_cli.models import BlockState, EventType, MemoryBlock, MemoryEvent - -# 创建内存块 -block = MemoryBlock( - id=1, - address=0x7f8b4c000000, - size=1024, - requested_size=1000, - state=BlockState.ACTIVE_ALLOCATED, - alloc_event_id=1, -) - -# 创建事件 -event = MemoryEvent( - id=1, - action=EventType.ALLOC, - address=block.address, - size=block.size, - stream=0, - allocated=1024, - active=1024, - reserved=2048, -) -``` - -## 相关文件 - -- [models/__init__.py](../../src/pt_snap_cli/models/__init__.py) - 模块导出 -- [models/block.py](../../src/pt_snap_cli/models/block.py) - MemoryBlock 实现 -- [models/event.py](../../src/pt_snap_cli/models/event.py) - MemoryEvent 实现 -- [models/_enums.py](../../src/pt_snap_cli/models/_enums.py) - 枚举定义 diff --git a/.agents/quickstart/04-query.md b/.agents/quickstart/04-query.md deleted file mode 100644 index 97d00ac..0000000 --- a/.agents/quickstart/04-query.md +++ /dev/null @@ -1,81 +0,0 @@ -# Query 模块 - -## 职责 - -提供内存快照的查询引擎,支持模板化查询和 Fluent API 构建自定义查询。 - -## 核心组件 - -### `QueryExecutor` -查询执行器,负责执行模板查询。 - -**方法:** -- `execute_template(template_name, params, device_id)` - 执行模板查询 -- `_load_template(template_name)` - 加载 YAML 模板 -- `_render_template(template, params)` - 渲染模板参数 - -### `QueryBuilder` -Fluent API 构建 SQL 查询。 - -**链式方法:** -- `from_table(table)` - 指定表 -- `columns(*cols)` - 选择列 -- `where(condition)` - 添加条件 -- `order_by(*cols, descending)` - 排序 -- `group_by(*cols)` - 分组 -- `limit(n)` / `offset(n)` - 分页 -- `build()` - 生成 SQL 字符串 - -### `Condition` -条件表达式构建器。 - -**方法:** -- `eq(value)` / `ne(value)` - 等于/不等于 -- `gt(value)` / `gte(value)` - 大于 -- `lt(value)` / `lte(value)` - 小于 -- `in_list(values)` - IN 查询 -- `like(pattern)` - LIKE 查询 -- `and_(other)` / `or_(other)` - 逻辑组合 - -### `QueryMapper` -将查询结果映射为 Python 对象。 - -### `Registry` -查询模板注册表,管理可用模板。 - -**函数:** -- `list_queries()` - 列出所有注册模板 - -## 依赖关系 - -- **依赖**:`Context` - 数据库连接 -- **依赖**:`Models` - 结果映射 -- **依赖**:`yaml` / `jinja2` - 模板渲染 -- **被依赖**:`cli.py` - CLI 调用 - -## 预定义模板 - -位于 `query/templates/` 目录,按分类组织: - -- **Basic**: `allocation`, `block`, `event` -- **Statistical**: `active_blocks_at_event`, `allocator_gap`, `callstack_analysis`, `memory_peak` -- **Business**: `active_memory_callstack_at_event`, `leak_detection` - -## 使用示例 - -```bash -pt-snap query snapshot.pkl.db --template-use leak_detection \ - --params '{"min_size": 1024}' --device 0 -pt-snap query --list -pt-snap query --template-info leak_detection -``` - -## 相关文件 - -- [query/__init__.py](../../src/pt_snap_cli/query/__init__.py) - 模块导出 -- [query/executor.py](../../src/pt_snap_cli/query/executor.py) - 执行器 -- [query/builder.py](../../src/pt_snap_cli/query/builder.py) - 构建器 -- [query/condition.py](../../src/pt_snap_cli/query/condition.py) - 条件 -- [query/mapper.py](../../src/pt_snap_cli/query/mapper.py) - 映射器 -- [query/registry.py](../../src/pt_snap_cli/query/registry.py) - 注册表 -- [query/templates/](../../src/pt_snap_cli/query/templates/) - 查询模板 diff --git a/.agents/quickstart/ARCHITECTURE.md b/.agents/quickstart/ARCHITECTURE.md deleted file mode 100644 index 47c303e..0000000 --- a/.agents/quickstart/ARCHITECTURE.md +++ /dev/null @@ -1,75 +0,0 @@ -# pt-snap-cli 架构概览 - -## 项目定位 - -PyTorch Memory Snapshot Analysis Tool - 用于分析 PyTorch 内存快照的命令行工具。 - -## 核心架构 - -``` -src/pt_snap_cli/ -├── cli.py # CLI 入口,提供 pt-snap 命令 -├── core/ # 共享服务与产品错误 -├── context.py # 只读数据库上下文 -├── snapshot/ # 内建快照导入与回放运行时 -├── models/ # 数据模型定义 -└── query/ # 查询引擎与模板 -``` - -## 模块依赖关系 - -``` -CLI (cli.py) - ↓ -Context (context.py) → SQLite 数据库连接 - ↓ -Query 模块 - ├── QueryExecutor # 查询执行器 - ├── QueryBuilder # SQL 构建器 - ├── QueryMapper # 结果映射器 - ├── Condition # 条件表达式 - └── Registry # 模板注册表 - ↓ -Models 模块 - ├── Block # 内存块 - └── Event # 内存事件 -``` - -## 数据流 - -1. **CLI 接收命令** → 解析参数 -2. **Context 加载数据库** → 只读模式连接 SQLite -3. **Query 执行查询** → 使用模板或自定义查询 -4. **Models 组织结果** → 结构化输出 - -## 关键设计决策 - -- **只读数据库访问**:确保分析过程不修改原始快照数据 -- **模板化查询**:预定义查询模板(leak_detection、callstack_analysis 等) -- **Fluent API**:QueryBuilder 提供链式调用接口 -- **设备隔离**:支持多 GPU 设备的数据分离 - -## 快速导航 - -- [CLI 模块详情](01-cli.md) - 命令行接口 -- [Context 模块详情](02-context.md) - 数据库上下文 -- [Models 模块详情](03-models.md) - 数据模型 -- [Query 模块详情](04-query.md) - 查询引擎 - -## 典型使用场景 - -```bash -# 1. 设置数据库 -pt-snap focus snapshot.pkl.db - -# 2. 执行泄漏检测 -pt-snap query snapshot.pkl.db --template-use leak_detection - -# 3. 调用栈分析 -pt-snap query snapshot.pkl.db --template-use callstack_analysis -``` - -## 相关文件 - -- [README.md](../../README.md) - 使用指南 -- [pyproject.toml](../../pyproject.toml) - 项目配置 diff --git a/.agents/quickstart/README-for-agents.md b/.agents/quickstart/README-for-agents.md deleted file mode 100644 index ae7c006..0000000 --- a/.agents/quickstart/README-for-agents.md +++ /dev/null @@ -1,88 +0,0 @@ -# Agent 文档使用指南 - -## 📚 文档结构 - -``` -.agents/quickstart/ -├── README-for-agents.md # 本文件 - 导航索引 -├── ARCHITECTURE.md # 整体架构概览 -├── 01-cli.md # CLI 模块详情 -├── 02-context.md # Context 模块详情 -├── 03-models.md # Models 模块详情 -└── 04-query.md # Query 模块详情 -``` - -## 🎯 阅读策略 - -### 新会话快速上手(推荐流程) - -1. **第一步**:阅读 [ARCHITECTURE.md](ARCHITECTURE.md) - - 了解项目定位和整体架构 - - 掌握模块依赖关系 - - 用时:2-3 分钟 - -2. **第二步**:根据任务需求选择阅读对应模块文档 - - CLI 相关 → [01-cli.md](01-cli.md) - - 数据库相关 → [02-context.md](02-context.md) - - 数据模型相关 → [03-models.md](03-models.md) - - 查询功能相关 → [04-query.md](04-query.md) - - 用时:每个模块 3-5 分钟 - -3. **第三步**:需要深入了解时再查阅源码 - -### 任务场景映射 - -| 任务类型 | 推荐文档 | -|---------|---------| -| 添加 CLI 命令 | ARCHITECTURE.md → 01-cli.md | -| 修改数据库操作 | ARCHITECTURE.md → 02-context.md | -| 新增数据模型 | ARCHITECTURE.md → 03-models.md | -| 开发查询功能 | ARCHITECTURE.md → 04-query.md | -| 修复 Bug | 根据错误定位选择对应模块文档 | -| 代码审查 | 全部模块文档快速浏览 | - -## 💡 渐进式披露原则 - -``` -ARCHITECTURE.md (5 分钟) - ↓ -模块文档 (3-5 分钟/个) - ↓ -源码细节 (按需深入) -``` - -**优势:** -- ✅ 新会话无需重新阅读全部源码 -- ✅ 快速建立认知地图 -- ✅ 按需深入,避免信息过载 -- ✅ 文档维护成本低于源码注释 - -## 📝 文档维护 - -当代码发生变更时: -1. 同步更新对应的模块文档 -2. 如有架构级变更,更新 ARCHITECTURE.md -3. 保持文档精简,只记录关键信息 - -## 🔗 与现有文档的关系 - -- `.agents/quickstart/` - **Agent 快速参考**(本文档) -- `README.md` - **用户使用指南** - -## ✨ 最佳实践 - -```markdown -# 新会话开场 -你好!我已阅读 [ARCHITECTURE.md](ARCHITECTURE.md), -了解到 pt-snap-cli 是一个 PyTorch 内存快照分析工具。 -请问需要我帮助你做什么? - -# 执行任务时 -根据任务需求,我会参考相关模块文档: -- 需要修改 CLI?→ 查看 01-cli.md -- 需要开发查询?→ 查看 04-query.md -``` - ---- - -**最后更新**: 2026-04-04 diff --git a/.gitignore b/.gitignore index 2d47bb0..4337dc4 100644 --- a/.gitignore +++ b/.gitignore @@ -1,7 +1,39 @@ +# --- Python / packaging --- +__pycache__/ +*.py[cod] +*.egg-info/ +build/ +dist/ +*.egg +.eggs/ + +# --- Tooling / test coverage --- +.coverage +.coverage.* +coverage.xml +htmlcov/ +.pytest_cache/ +.ruff_cache/ +.mypy_cache/ +.tox/ + +# --- Local agent/tool scratch (not project source) --- +.agents/* +.claude/* +.codex/* +.opencode/* +.tmp/* +openspec/ +specs/ + +# --- OS cruft --- +.DS_Store + +# --- superpowers / git worktrees (project-local convention) --- **/superpowers/ -.claude/worktrees/ +.worktrees/ -# Sensitive test data - do not commit +# --- Sensitive test data - do not commit --- tests/fixtures/snapshots/*.pickle !tests/fixtures/snapshots/snapshot_import_131k_sanitized.pickle !tests/fixtures/snapshots/snapshot_import_628k_sanitized.pickle diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..1f93e91 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,109 @@ +# AGENTS.md + +This is the repository-wide entry point for coding agents. Keep durable project +facts here; user-facing behavior belongs in `README.md` and `docs/`. + +## Project Scope + +`pt-snap-cli` is a Python 3.10+ package for importing, splitting, inspecting, +and querying PyTorch memory snapshots. It exposes a Typer CLI, a Python API, +and an MCP server over shared service-layer behavior. + +## Development Commands + +Run these commands from the repository root. + +- Install for local development: `pip install -e .` +- Install development dependencies: `pip install -e .[dev]` +- Run all tests: `pytest` +- Run one file: `pytest tests/test_cli.py` +- Run one test: `pytest tests/test_cli.py -k test_version_flag` +- Lint: `ruff check .` +- Check formatting: `black --check .` +- Apply formatting: `black .` + +`tests/run_tests.sh` is tied to a developer-specific Conda path and writes +coverage reports under `test_reports/`; use direct `pytest` commands unless +that local environment is intentionally available. + +## Runtime Topology + +| Surface | Entry point | Responsibility | +| --- | --- | --- | +| CLI | `src/pt_snap_cli/cli.py` via `pt_snap_cli.cli:_safe_call` | Typer commands for focus, import, split, metadata, query, reports, and config | +| Python API | `src/pt_snap_cli/api.py` (`SnapshotAnalyzer`) | Programmatic focus, query, and metadata facade | +| MCP | `src/pt_snap_cli/mcp/server.py` via `pt_snap_cli.mcp.server:main` | Agent tools/resources backed by `SnapshotAnalyzer` | +| Product services | `src/pt_snap_cli/core/` | Shared focus, import, split, query, report, metadata, and error semantics | +| Database access | `src/pt_snap_cli/context.py` | Read-only SQLite validation, connection management, and device discovery | +| Query engine | `src/pt_snap_cli/query/` | YAML loading, registry, parameter validation, SQL rendering/execution, and result mapping | +| Snapshot runtime | `src/pt_snap_cli/snapshot/` | First-party pickle/JSON representation, replay, database import, and slicing | +| Library models | `src/pt_snap_cli/models/` | Dataclass models and allocator event/block enums | + +Installed scripts and package data are declared in `pyproject.toml`. Query +templates under category subdirectories are included by +`query/templates/*/*.yaml`. + +## Repository Invariants + +### CLI, API, and MCP + +- Keep normalized CLI and MCP behavior in shared API/core services; adapters may + format output differently but must not redefine focus, query, metadata, or + error semantics. +- Update `tests/test_contract_cli_mcp.py` when a shared CLI/MCP capability or + normalized result changes. + +### Focus and database selection + +- `Config.resolve_focus()` resolves the database in this order: explicit path, + `PT_SNAP_DB_PATH`, nearest ancestor `.pt-snap/focus.json`, then legacy global + config at `~/.config/pt-snap-cli/config.json`. +- Query device selection is explicit device, focused device, then the first + device discovered from `trace_entry_` tables. +- `Context` opens SQLite with `mode=ro` and validates the `dictionary` table. + Do not introduce writes into the analysis path. + +### Query templates + +- Packaged templates live under `src/pt_snap_cli/query/templates//*.yaml`. +- `query/config.py` owns YAML parsing and parameter validation; + `query/registry.py` recursively loads templates into the singleton registry; + `query/executor.py` renders with Jinja2 `StrictUndefined` and injects device + table names before executing through `Context`. +- When template metadata or behavior changes, review the YAML, config, registry, + executor, `core/query_service.py`, CLI/MCP presentation, and focused tests + together. + +### Snapshot import and split + +- Pickle input is trusted-code execution, not a sandbox. Never load an + untrusted fixture or snapshot merely to inspect it. +- Import writes and validates metadata on a temporary database before + `os.replace()` publishes it. Preserve the existing destination on failure. +- Split requires exactly one positive strategy (`--slices` or + `--max-entries`), an absent destination, replay-validates generated slices, + and publishes the staged directory without replacement. +- `src/pt_snap_cli/snapshot/PROVENANCE.md` is append-only. Changes below the + snapshot runtime require the provenance decision enforced by + `.github/scripts/check_snapshot_provenance.py` and covered by + `tests/test_governance.py`. + +## Change Routing + +| Change | Start here | Focused tests | +| --- | --- | --- | +| CLI options or terminal rendering | `src/pt_snap_cli/cli.py` and the owning core service | `tests/test_cli.py`, `tests/test_completion.py` | +| Focus precedence or persistence | `src/pt_snap_cli/config.py`, `src/pt_snap_cli/core/focus_service.py`, `src/pt_snap_cli/context.py` | `tests/test_config.py`, `tests/test_context.py`, `tests/core/test_focus_service.py` | +| Query schema, SQL, or categories | `src/pt_snap_cli/query/`, `src/pt_snap_cli/core/query_service.py` | `tests/query/`, `tests/core/test_query_service.py` | +| MCP or Python API behavior | `src/pt_snap_cli/api.py`, `src/pt_snap_cli/mcp/server.py` | `tests/test_api.py`, `tests/test_mcp_server.py`, `tests/test_contract_cli_mcp.py` | +| Snapshot import or metadata | `src/pt_snap_cli/core/import_service.py`, `src/pt_snap_cli/core/import_metadata.py`, `src/pt_snap_cli/core/snapshot_import_backend.py` | `tests/core/test_import_*.py`, `tests/test_snapshot_db.py` | +| Snapshot splitting or replay | `src/pt_snap_cli/core/split_service.py`, `src/pt_snap_cli/snapshot/` | `tests/core/test_split_service.py`, `tests/snapshot/` | +| Reports | `src/pt_snap_cli/core/report_service.py`, report commands in `src/pt_snap_cli/cli.py` | `tests/core/test_report_service.py`, report cases in `tests/test_cli.py` | +| Packaging or release | `pyproject.toml`, `.github/workflows/` | `tests/test_package.py`, `tests/test_release_workflow.py` | + +## Documentation Boundaries + +- `README.md` and `README_zh.md` are user-facing entry points. +- `docs/en/` and `docs/zh/` must remain behaviorally aligned for user guides. +- Keep agent instructions in this file unless a subtree gains an independent + lifecycle or maintenance contract that warrants a local `AGENTS.md`. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..43c994c --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +@AGENTS.md