From 3f1950d70338f002a8b3609c6e9899675537cf75 Mon Sep 17 00:00:00 2001 From: codeasier Date: Sat, 8 Aug 2026 01:42:54 +0800 Subject: [PATCH 1/3] docs: add AGENTS.md guidance across repo subtrees --- .github/AGENTS.md | 43 ++++++++++++++ .github/ISSUE_TEMPLATE/AGENTS.md | 43 ++++++++++++++ .github/PULL_REQUEST_TEMPLATE/AGENTS.md | 39 +++++++++++++ .github/workflows/AGENTS.md | 43 ++++++++++++++ docs/AGENTS.md | 44 ++++++++++++++ docs/en/AGENTS.md | 45 +++++++++++++++ docs/zh/AGENTS.md | 45 +++++++++++++++ src/AGENTS.md | 39 +++++++++++++ src/pt_snap_cli/AGENTS.md | 57 +++++++++++++++++++ src/pt_snap_cli/core/AGENTS.md | 48 ++++++++++++++++ src/pt_snap_cli/mcp/AGENTS.md | 43 ++++++++++++++ src/pt_snap_cli/models/AGENTS.md | 42 ++++++++++++++ src/pt_snap_cli/query/AGENTS.md | 52 +++++++++++++++++ src/pt_snap_cli/query/templates/AGENTS.md | 46 +++++++++++++++ .../query/templates/basic/AGENTS.md | 42 ++++++++++++++ .../query/templates/business/AGENTS.md | 40 +++++++++++++ .../query/templates/statistical/AGENTS.md | 41 +++++++++++++ tests/AGENTS.md | 53 +++++++++++++++++ tests/core/AGENTS.md | 41 +++++++++++++ tests/models/AGENTS.md | 42 ++++++++++++++ tests/query/AGENTS.md | 48 ++++++++++++++++ 21 files changed, 936 insertions(+) create mode 100644 .github/AGENTS.md create mode 100644 .github/ISSUE_TEMPLATE/AGENTS.md create mode 100644 .github/PULL_REQUEST_TEMPLATE/AGENTS.md create mode 100644 .github/workflows/AGENTS.md create mode 100644 docs/AGENTS.md create mode 100644 docs/en/AGENTS.md create mode 100644 docs/zh/AGENTS.md create mode 100644 src/AGENTS.md create mode 100644 src/pt_snap_cli/AGENTS.md create mode 100644 src/pt_snap_cli/core/AGENTS.md create mode 100644 src/pt_snap_cli/mcp/AGENTS.md create mode 100644 src/pt_snap_cli/models/AGENTS.md create mode 100644 src/pt_snap_cli/query/AGENTS.md create mode 100644 src/pt_snap_cli/query/templates/AGENTS.md create mode 100644 src/pt_snap_cli/query/templates/basic/AGENTS.md create mode 100644 src/pt_snap_cli/query/templates/business/AGENTS.md create mode 100644 src/pt_snap_cli/query/templates/statistical/AGENTS.md create mode 100644 tests/AGENTS.md create mode 100644 tests/core/AGENTS.md create mode 100644 tests/models/AGENTS.md create mode 100644 tests/query/AGENTS.md diff --git a/.github/AGENTS.md b/.github/AGENTS.md new file mode 100644 index 0000000..3efdc01 --- /dev/null +++ b/.github/AGENTS.md @@ -0,0 +1,43 @@ + + + +# .github + +## Purpose +`.github` contains GitHub project automation: issue forms, pull request templates, release workflow, and CI test/type-check workflow definitions. + +## Key Files +| File | Description | +|------|-------------| +| None | GitHub metadata is organized in subdirectories. | + +## Subdirectories +| Directory | Purpose | +|-----------|---------| +| `ISSUE_TEMPLATE/` | Structured issue form YAML files (see `ISSUE_TEMPLATE/AGENTS.md`). | +| `PULL_REQUEST_TEMPLATE/` | Pull request description template (see `PULL_REQUEST_TEMPLATE/AGENTS.md`). | +| `workflows/` | GitHub Actions workflow definitions (see `workflows/AGENTS.md`). | + +## For AI Agents + +### Working In This Directory +- Treat workflow changes as CI-affecting; keep dependency installation and verification commands aligned with `pyproject.toml`. +- Keep issue and PR templates concise and consistent with repository scope. + +### Testing Requirements +- Validate YAML syntax for workflow or issue template edits. +- For workflow changes, mirror commands locally where practical before relying on CI. + +### Common Patterns +- CI installs the package with development dependencies and runs lint/type/test checks. + +## Dependencies + +### Internal +- `pyproject.toml` defines the Python versions, package dependencies, and tools invoked by workflows. +- `tests/` contains the suite run by CI. + +### External +- GitHub Actions runners and standard GitHub issue/PR template formats. + + diff --git a/.github/ISSUE_TEMPLATE/AGENTS.md b/.github/ISSUE_TEMPLATE/AGENTS.md new file mode 100644 index 0000000..e785ab3 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/AGENTS.md @@ -0,0 +1,43 @@ + + + +# ISSUE_TEMPLATE + +## Purpose +`ISSUE_TEMPLATE` contains structured GitHub issue forms for bugs, documentation requests, feature requests, questions, and template chooser configuration. + +## Key Files +| File | Description | +|------|-------------| +| `bug_report.yaml` | Form for reproducible defects and environment details. | +| `config.yaml` | GitHub issue template chooser configuration. | +| `documentation.yaml` | Form for documentation improvements or corrections. | +| `feature_request.yaml` | Form for proposed enhancements. | +| `question.yaml` | Form for usage questions. | + +## Subdirectories +| Directory | Purpose | +|-----------|---------| +| None | Issue templates are flat YAML files. | + +## For AI Agents + +### Working In This Directory +- Keep issue forms scoped to this Python CLI/MCP project. +- Preserve valid GitHub issue form YAML structure. + +### Testing Requirements +- Validate YAML syntax after edits. + +### Common Patterns +- Forms collect structured fields rather than relying on free-form Markdown only. + +## Dependencies + +### Internal +- Repository README/docs provide canonical terminology for issue prompts. + +### External +- GitHub issue form YAML schema. + + diff --git a/.github/PULL_REQUEST_TEMPLATE/AGENTS.md b/.github/PULL_REQUEST_TEMPLATE/AGENTS.md new file mode 100644 index 0000000..4494f37 --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE/AGENTS.md @@ -0,0 +1,39 @@ + + + +# PULL_REQUEST_TEMPLATE + +## Purpose +`PULL_REQUEST_TEMPLATE` contains the default pull request description template used for repository contributions. + +## Key Files +| File | Description | +|------|-------------| +| `pull_request_template.md` | PR body template for summaries, testing, and review context. | + +## Subdirectories +| Directory | Purpose | +|-----------|---------| +| None | PR template content is flat. | + +## For AI Agents + +### Working In This Directory +- Keep checklist items aligned with current verification expectations. +- Avoid adding process requirements that CI or project documentation does not support. + +### Testing Requirements +- Review Markdown rendering after edits. + +### Common Patterns +- The template should prompt contributors for what changed and how it was tested. + +## Dependencies + +### Internal +- `.github/workflows/` defines automated checks referenced by PR guidance. + +### External +- GitHub pull request template Markdown support. + + diff --git a/.github/workflows/AGENTS.md b/.github/workflows/AGENTS.md new file mode 100644 index 0000000..2817ddf --- /dev/null +++ b/.github/workflows/AGENTS.md @@ -0,0 +1,43 @@ + + + +# workflows + +## Purpose +`workflows` contains GitHub Actions definitions for continuous integration and release automation. + +## Key Files +| File | Description | +|------|-------------| +| `test.yml` | CI workflow for installing dev dependencies and running lint/type/test checks. | +| `release.yml` | Release workflow for building and publishing package artifacts. | + +## Subdirectories +| Directory | Purpose | +|-----------|---------| +| None | Workflow YAML files are flat. | + +## For AI Agents + +### Working In This Directory +- Keep workflow Python versions and commands aligned with `pyproject.toml`. +- Treat release workflow edits as high-impact and verify syntax carefully. +- Do not bypass tests, lint, type checks, or publishing safeguards without explicit instruction. + +### Testing Requirements +- Validate YAML syntax. +- Run equivalent local commands when changing CI command sequences. + +### Common Patterns +- Workflows install the package with development extras before verification. + +## Dependencies + +### Internal +- `pyproject.toml` defines package metadata, dev extras, and tool configuration. +- `tests/` and source files are exercised by CI. + +### External +- GitHub Actions hosted runners and packaging/publishing actions. + + diff --git a/docs/AGENTS.md b/docs/AGENTS.md new file mode 100644 index 0000000..20214f6 --- /dev/null +++ b/docs/AGENTS.md @@ -0,0 +1,44 @@ + + + +# docs + +## Purpose +`docs` contains end-user and API documentation for the CLI, focus management, query templates, MCP integration, snapshot database format, and result mapping API. It is split into English and Chinese language trees with a top-level README for navigation. + +## Key Files +| File | Description | +|------|-------------| +| `README.md` | Documentation landing page and language/topic navigation. | + +## Subdirectories +| Directory | Purpose | +|-----------|---------| +| `en/` | English documentation guides (see `en/AGENTS.md`). | +| `zh/` | Chinese documentation guides (see `zh/AGENTS.md`). | + +## For AI Agents + +### Working In This Directory +- Keep English and Chinese docs synchronized when changing user-visible behavior. +- Verify command names, options, template names, and MCP tool names against source code before updating docs. +- Prefer updating existing guides over creating new documentation pages. + +### Testing Requirements +- For documentation-only changes, review rendered Markdown structure and verify referenced commands/options exist. +- If examples depend on behavior changes, run the matching CLI or service tests. + +### Common Patterns +- The two language trees mirror the same topics: quick start, focus management, querying, MCP, database schema, and result mapper API. + +## Dependencies + +### Internal +- `src/pt_snap_cli/cli.py` defines documented CLI commands and options. +- `src/pt_snap_cli/mcp/server.py` defines documented MCP tools, resource, and prompt. +- `src/pt_snap_cli/query/templates/` defines documented built-in template names and parameters. + +### External +- Markdown is the only documentation format used here. + + diff --git a/docs/en/AGENTS.md b/docs/en/AGENTS.md new file mode 100644 index 0000000..3dec0d4 --- /dev/null +++ b/docs/en/AGENTS.md @@ -0,0 +1,45 @@ + + + +# en + +## Purpose +`docs/en` contains the English documentation set for installing and using `pt-snap-cli`, managing focus, querying snapshot databases, running the MCP server, understanding the SQLite schema, and using result mapping APIs. + +## Key Files +| File | Description | +|------|-------------| +| `quickstart.md` | End-to-end installation and first-query walkthrough. | +| `focus-management.md` | Focus resolution, project focus files, environment variables, and global config behavior. | +| `querying.md` | Query listing, template info, parameter usage, result limits, and built-in templates. | +| `mcp.md` | MCP server setup, tools, resources, prompts, and agent usage. | +| `database.md` | Snapshot SQLite table/schema reference. | +| `result-mapper-api.md` | Result mapping API documentation. | + +## Subdirectories +| Directory | Purpose | +|-----------|---------| +| None | English docs are flat topic files. | + +## For AI Agents + +### Working In This Directory +- Keep examples executable against current CLI options and template names. +- Mirror user-visible changes in `../zh/` when appropriate. + +### Testing Requirements +- Verify referenced commands and options against `src/pt_snap_cli/cli.py`. +- Verify template details against `src/pt_snap_cli/query/templates/`. + +### Common Patterns +- Topic files correspond to major product areas rather than source modules. + +## Dependencies + +### Internal +- CLI, MCP, query template, and schema details should be checked against `src/pt_snap_cli/` before editing. + +### External +- Markdown only. + + diff --git a/docs/zh/AGENTS.md b/docs/zh/AGENTS.md new file mode 100644 index 0000000..a2b5f88 --- /dev/null +++ b/docs/zh/AGENTS.md @@ -0,0 +1,45 @@ + + + +# zh + +## Purpose +`docs/zh` contains the Chinese documentation set for `pt-snap-cli`, covering the same user-facing areas as the English docs: quick start, focus management, querying, MCP integration, database schema, and result mapping API. + +## Key Files +| File | Description | +|------|-------------| +| `quickstart.md` | Chinese installation and first-query walkthrough. | +| `focus-management.md` | Chinese guide to focus resolution and persistence. | +| `querying.md` | Chinese guide to query templates, parameters, and output. | +| `mcp.md` | Chinese MCP server setup and tool reference. | +| `database.md` | Chinese snapshot SQLite schema reference. | +| `result-mapper-api.md` | Chinese result mapping API guide. | + +## Subdirectories +| Directory | Purpose | +|-----------|---------| +| None | Chinese docs are flat topic files. | + +## For AI Agents + +### Working In This Directory +- Keep translations aligned with `../en/` and current source behavior. +- Preserve Chinese terminology consistently across guides. + +### Testing Requirements +- Verify commands, options, template names, and MCP tools against source before editing examples. + +### Common Patterns +- Files mirror the English documentation topics one-for-one. + +## Dependencies + +### Internal +- `../en/` is the sibling language reference for topic coverage. +- `src/pt_snap_cli/` is the behavior source of truth. + +### External +- Markdown only. + + diff --git a/src/AGENTS.md b/src/AGENTS.md new file mode 100644 index 0000000..e35a3eb --- /dev/null +++ b/src/AGENTS.md @@ -0,0 +1,39 @@ + + + +# src + +## Purpose +`src` contains the installable Python package for `pt-snap-cli`. Its only package subtree, `pt_snap_cli`, holds the CLI, MCP server, focus and database services, query system, and domain models shipped in the distribution. + +## Key Files +| File | Description | +|------|-------------| +| None | Source files live under the `pt_snap_cli/` package. | + +## Subdirectories +| Directory | Purpose | +|-----------|---------| +| `pt_snap_cli/` | Main package implementation (see `pt_snap_cli/AGENTS.md`). | + +## For AI Agents + +### Working In This Directory +- Keep import paths package-relative to `pt_snap_cli` and avoid adding sibling packages unless packaging metadata is updated. +- When adding package data, update `pyproject.toml` so files are included in builds. + +### Testing Requirements +- Run package-level tests from the repository root with `pytest` or focused tests in `tests/`. + +### Common Patterns +- The repository uses a `src/` layout, so tests and local commands should import the installed/editable package rather than relying on the repository root being importable. + +## Dependencies + +### Internal +- `pt_snap_cli/` is installed via the `pyproject.toml` setuptools configuration. + +### External +- setuptools package discovery uses `where = ["src"]`. + + diff --git a/src/pt_snap_cli/AGENTS.md b/src/pt_snap_cli/AGENTS.md new file mode 100644 index 0000000..7aa4ad3 --- /dev/null +++ b/src/pt_snap_cli/AGENTS.md @@ -0,0 +1,57 @@ + + + +# pt_snap_cli + +## Purpose +`pt_snap_cli` is the main Python package. It exposes the Typer CLI, programmatic `SnapshotAnalyzer` API, read-only SQLite context, focus configuration, completion helpers, service layer, MCP server, domain models, and YAML-driven query execution pipeline. + +## Key Files +| File | Description | +|------|-------------| +| `__init__.py` | Package exports and version wiring. | +| `api.py` | High-level `SnapshotAnalyzer` API used by MCP and library callers. | +| `cli.py` | Typer CLI entrypoint for `pt-snap focus`, `pt-snap query`, and `pt-snap config`. | +| `completion.py` | Shell completion helpers for templates, categories, and device IDs. | +| `config.py` | Focus persistence and resolution across explicit paths, environment, project focus, and legacy global config. | +| `context.py` | Read-only SQLite context with schema validation and device discovery. | +| `version.py` | Package version constant. | + +## Subdirectories +| Directory | Purpose | +|-----------|---------| +| `core/` | Service-layer models and typed domain errors (see `core/AGENTS.md`). | +| `mcp/` | FastMCP server entrypoint and tools (see `mcp/AGENTS.md`). | +| `models/` | Domain models for snapshot blocks, events, and enums (see `models/AGENTS.md`). | +| `query/` | Query builders, config loading, execution, mapping, registry, and templates (see `query/AGENTS.md`). | + +## For AI Agents + +### Working In This Directory +- Route user-facing command logic through `core` services when practical so CLI, API, and MCP behavior remain aligned. +- Preserve focus precedence: explicit CLI/API path, `PT_SNAP_DB_PATH`, nearest project `.pt-snap/focus.json`, then legacy global config. +- Keep SQLite access read-only for analysis paths. + +### Testing Requirements +- CLI changes should update/run `tests/test_cli.py` and any affected config/focus tests. +- API or MCP changes should update/run `tests/test_api.py` and `tests/test_mcp_server.py`. +- Context or config changes should update/run `tests/test_context.py` and `tests/test_config.py`. + +### Common Patterns +- Exceptions from lower-level modules are translated into `core.errors` before reaching CLI/service callers. +- Query template metadata shown to users comes from the registry/config pipeline, not hardcoded CLI text. + +## Dependencies + +### Internal +- `core/` coordinates focus and query behavior for CLI/API callers. +- `query/` provides template discovery and execution. +- `models/` contains library-facing snapshot structures. + +### External +- `typer` for CLI commands and output. +- `sqlite3` standard library for database access. +- `mcp` for agent server integration. +- `jinja2` and `pyyaml` through the query subsystem. + + diff --git a/src/pt_snap_cli/core/AGENTS.md b/src/pt_snap_cli/core/AGENTS.md new file mode 100644 index 0000000..965a3ff --- /dev/null +++ b/src/pt_snap_cli/core/AGENTS.md @@ -0,0 +1,48 @@ + + + +# core + +## Purpose +`core` contains the service layer shared by the CLI and public API. It wraps lower-level config, context, and query modules with stable typed models and domain-specific exceptions for focus management and query execution. + +## Key Files +| File | Description | +|------|-------------| +| `__init__.py` | Re-exports service classes, models, and errors for package consumers. | +| `errors.py` | Domain exception types used by CLI/API service boundaries. | +| `focus_service.py` | Focus resolution, validation, project/global focus writes, and device selection. | +| `models.py` | Dataclasses for focus state, resolved focus, template metadata, and query results. | +| `query_service.py` | Template listing/info and query execution orchestration using resolved focus and `QueryExecutor`. | + +## Subdirectories +| Directory | Purpose | +|-----------|---------| +| None | Service-layer modules are flat. | + +## For AI Agents + +### Working In This Directory +- Translate low-level exceptions into `core.errors` so CLI and API callers receive consistent failures. +- Preserve explicit device precedence over focused device, and validate devices against `Context.device_ids`. +- Keep service models stable when changing CLI/API/MCP output shapes. + +### Testing Requirements +- Run `pytest tests/core` for service changes. +- Also run CLI/API tests when service behavior changes user-visible output or errors. + +### Common Patterns +- `FocusService` delegates persistence to `Config` and validation to `Context`. +- `QueryService` resolves focus, chooses a target device, executes a named template, and applies row limiting. + +## Dependencies + +### Internal +- `pt_snap_cli.config` for focus persistence and precedence. +- `pt_snap_cli.context` for database validation and device discovery. +- `pt_snap_cli.query` for template registry and SQL execution. + +### External +- `sqlite3` standard library exceptions are normalized at service boundaries. + + diff --git a/src/pt_snap_cli/mcp/AGENTS.md b/src/pt_snap_cli/mcp/AGENTS.md new file mode 100644 index 0000000..81a7e1d --- /dev/null +++ b/src/pt_snap_cli/mcp/AGENTS.md @@ -0,0 +1,43 @@ + + + +# mcp + +## Purpose +`mcp` exposes `pt-snap-cli` functionality as a FastMCP server so agents and MCP clients can inspect or set focus, list templates, fetch template metadata, execute queries, and use a memory-leak analysis prompt. + +## Key Files +| File | Description | +|------|-------------| +| `__init__.py` | MCP package marker. | +| `server.py` | FastMCP app, tools, resource, prompt, and `pt-snap-mcp` entrypoint. | + +## Subdirectories +| Directory | Purpose | +|-----------|---------| +| None | MCP implementation is contained in `server.py`. | + +## For AI Agents + +### Working In This Directory +- Keep MCP tool behavior aligned with `SnapshotAnalyzer` rather than duplicating CLI-specific logic. +- Use JSON-serializable return values for tools and resources. +- Update docs in `docs/*/mcp.md` and tests when adding or changing MCP tools. + +### Testing Requirements +- Run `pytest tests/test_mcp_server.py` for MCP changes. +- Run API tests if MCP changes require `SnapshotAnalyzer` changes. + +### Common Patterns +- A module-level `SnapshotAnalyzer` backs all MCP tools. +- `focus://current` mirrors the current focus tool response. + +## Dependencies + +### Internal +- `pt_snap_cli.api.SnapshotAnalyzer` provides the server-facing API. + +### External +- `mcp.server.fastmcp.FastMCP` provides tool/resource/prompt registration and server execution. + + diff --git a/src/pt_snap_cli/models/AGENTS.md b/src/pt_snap_cli/models/AGENTS.md new file mode 100644 index 0000000..bf64046 --- /dev/null +++ b/src/pt_snap_cli/models/AGENTS.md @@ -0,0 +1,42 @@ + + + +# models + +## Purpose +`models` contains library-facing domain objects for PyTorch memory snapshot data, including allocation blocks, events, and enum values that describe allocator state and event types. + +## Key Files +| File | Description | +|------|-------------| +| `__init__.py` | Public model exports. | +| `_enums.py` | Enum definitions for snapshot concepts. | +| `block.py` | Block-related data model. | +| `event.py` | Event-related data model. | + +## Subdirectories +| Directory | Purpose | +|-----------|---------| +| None | Domain models are flat. | + +## For AI Agents + +### Working In This Directory +- Keep model fields aligned with the snapshot schema and query output mapping expectations. +- Update exports in `__init__.py` when adding public models. + +### Testing Requirements +- Run `pytest tests/models` and any package-level model tests after changing models or enums. + +### Common Patterns +- Models are used by library-facing code more than the CLI, which usually prints plain dictionaries from query execution. + +## Dependencies + +### Internal +- Query result mapping may depend on model-compatible types and enum values. + +### External +- Standard Python dataclass/enum-style modeling; no heavy external model framework is used here. + + diff --git a/src/pt_snap_cli/query/AGENTS.md b/src/pt_snap_cli/query/AGENTS.md new file mode 100644 index 0000000..5332ddc --- /dev/null +++ b/src/pt_snap_cli/query/AGENTS.md @@ -0,0 +1,52 @@ + + + +# query + +## Purpose +`query` implements the template-driven query subsystem. It loads YAML query definitions, validates template parameters, renders device-specific SQL with Jinja2, executes queries through the read-only database context, maps result types, and provides fluent builder utilities for constructing SQL programmatically. + +## Key Files +| File | Description | +|------|-------------| +| `__init__.py` | Query package marker/exports. | +| `builder.py` | Fluent `QueryBuilder` for SELECT statements with conditions, grouping, ordering, limits, and offsets. | +| `condition.py` | Composable SQL condition objects that emit parameterized SQL fragments. | +| `config.py` | YAML query config loader and `QueryTemplate`/`QueryParameter` validation. | +| `executor.py` | Jinja2 SQL rendering and query execution through `Context`. | +| `mapper.py` | Result type conversion and optional model-factory mapping. | +| `registry.py` | Singleton registry that loads packaged YAML templates and exposes listing/lookup helpers. | + +## Subdirectories +| Directory | Purpose | +|-----------|---------| +| `templates/` | Built-in YAML query templates grouped by category (see `templates/AGENTS.md`). | + +## For AI Agents + +### Working In This Directory +- When changing template behavior, check YAML templates, `config.py`, `registry.py`, `executor.py`, and CLI template metadata output together. +- Keep SQL value filters parameterized where using builder/condition APIs. +- Device-specific template SQL should use injected table names such as `device_trace_table` and `device_block_table`. + +### Testing Requirements +- Run `pytest tests/query` for query subsystem changes. +- Run `pytest tests/test_cli.py` when template listing/info/output behavior changes. + +### Common Patterns +- Registry lookup is global and package templates load at import time. +- `StrictUndefined` is used during template rendering so missing template variables fail loudly. +- Template categories are inferred from directory structure when not explicitly declared in YAML. + +## Dependencies + +### Internal +- `pt_snap_cli.context.Context` provides database connections and device discovery. +- `src/pt_snap_cli/query/templates/` provides built-in templates. +- `core.query_service` consumes registry and executor APIs. + +### External +- `pyyaml` for YAML parsing. +- `jinja2` for SQL template rendering. + + diff --git a/src/pt_snap_cli/query/templates/AGENTS.md b/src/pt_snap_cli/query/templates/AGENTS.md new file mode 100644 index 0000000..38e0d5a --- /dev/null +++ b/src/pt_snap_cli/query/templates/AGENTS.md @@ -0,0 +1,46 @@ + + + +# templates + +## Purpose +`templates` contains the packaged YAML query templates used by the registry and CLI/API/MCP query execution. Templates are grouped by category and define metadata, parameters, SQL, and output schemas. + +## Key Files +| File | Description | +|------|-------------| +| None | Template YAML files are grouped in category subdirectories. | + +## Subdirectories +| Directory | Purpose | +|-----------|---------| +| `basic/` | Core allocation/block/event inspection templates (see `basic/AGENTS.md`). | +| `business/` | Higher-level analysis templates such as leak detection (see `business/AGENTS.md`). | +| `statistical/` | Aggregation and statistical analysis templates (see `statistical/AGENTS.md`). | + +## For AI Agents + +### Working In This Directory +- Keep each YAML template's `queries` entry name aligned with the file/topic and documented template name. +- Include accurate parameter defaults, required flags, descriptions, and output schema entries. +- Use Jinja variables provided by `QueryExecutor`, especially device-specific table names. + +### Testing Requirements +- Run `pytest tests/query/test_registry.py tests/query/test_config.py tests/query/test_executor.py` after template changes. +- Run `pytest tests/test_cli.py` if list or template-info output changes. + +### Common Patterns +- Category is inferred from the subdirectory unless explicitly set in the YAML. +- SQL is rendered through Jinja2 before execution against SQLite. + +## Dependencies + +### Internal +- `pt_snap_cli.query.config.QueryConfig` parses template YAML. +- `pt_snap_cli.query.registry` discovers and registers packaged templates. +- `pt_snap_cli.query.executor.QueryExecutor` renders and executes template SQL. + +### External +- YAML syntax via `pyyaml` and Jinja2 expressions inside SQL strings. + + diff --git a/src/pt_snap_cli/query/templates/basic/AGENTS.md b/src/pt_snap_cli/query/templates/basic/AGENTS.md new file mode 100644 index 0000000..24f3e54 --- /dev/null +++ b/src/pt_snap_cli/query/templates/basic/AGENTS.md @@ -0,0 +1,42 @@ + + + +# basic templates + +## Purpose +`basic` contains core query templates for inspecting low-level allocation, block, and event records in a focused snapshot database. + +## Key Files +| File | Description | +|------|-------------| +| `allocation.yaml` | Allocation-oriented query template. | +| `block.yaml` | Block inspection query template. | +| `event.yaml` | Event timeline/query template. | + +## Subdirectories +| Directory | Purpose | +|-----------|---------| +| None | Basic templates are flat YAML files. | + +## For AI Agents + +### Working In This Directory +- Keep templates narrowly focused on direct snapshot inspection. +- Ensure parameter descriptions and output schemas match the SQL result columns. + +### Testing Requirements +- Run query config/registry/executor tests after changing YAML. +- Run CLI template-info tests if metadata changes. + +### Common Patterns +- Templates query device-specific trace or block tables using injected table names. + +## Dependencies + +### Internal +- Loaded by `pt_snap_cli.query.registry` and executed by `QueryExecutor`. + +### External +- YAML plus Jinja2 template syntax inside SQL. + + diff --git a/src/pt_snap_cli/query/templates/business/AGENTS.md b/src/pt_snap_cli/query/templates/business/AGENTS.md new file mode 100644 index 0000000..54a329c --- /dev/null +++ b/src/pt_snap_cli/query/templates/business/AGENTS.md @@ -0,0 +1,40 @@ + + + +# business templates + +## Purpose +`business` contains higher-level analysis templates that package common memory investigation workflows into named queries. + +## Key Files +| File | Description | +|------|-------------| +| `leak_detection.yaml` | Template for identifying potential memory leaks using allocation-size thresholds. | + +## Subdirectories +| Directory | Purpose | +|-----------|---------| +| None | Business templates are flat YAML files. | + +## For AI Agents + +### Working In This Directory +- Keep query names and descriptions understandable to CLI users rather than implementation-oriented. +- Validate that defaults are safe for broad snapshot exploration. + +### Testing Requirements +- Run query registry/config/executor tests after changing templates. +- Run docs/CLI checks if user-facing metadata changes. + +### Common Patterns +- Templates may expose business-level parameters such as minimum allocation size. + +## Dependencies + +### Internal +- Consumed by query registry, CLI template listing, API/MCP template tools, and `QueryExecutor`. + +### External +- YAML plus Jinja2 template syntax inside SQL. + + diff --git a/src/pt_snap_cli/query/templates/statistical/AGENTS.md b/src/pt_snap_cli/query/templates/statistical/AGENTS.md new file mode 100644 index 0000000..64ea3c0 --- /dev/null +++ b/src/pt_snap_cli/query/templates/statistical/AGENTS.md @@ -0,0 +1,41 @@ + + + +# statistical templates + +## Purpose +`statistical` contains aggregation-oriented query templates for summarizing memory peaks, callstack behavior, and other statistical views of snapshot data. + +## Key Files +| File | Description | +|------|-------------| +| `callstack_analysis.yaml` | Template for grouping or analyzing memory behavior by callstack. | +| `memory_peak.yaml` | Template for identifying peak memory usage. | + +## Subdirectories +| Directory | Purpose | +|-----------|---------| +| None | Statistical templates are flat YAML files. | + +## For AI Agents + +### Working In This Directory +- Keep aggregation result columns documented in each template's output schema. +- Be careful with SQL grouping and ordering so CLI output remains deterministic. + +### Testing Requirements +- Run query executor and registry tests after changing templates. +- Add fixture coverage when introducing new aggregation assumptions. + +### Common Patterns +- Templates summarize many low-level records into ranked or grouped result rows. + +## Dependencies + +### Internal +- Consumed by query registry, CLI/API/MCP template metadata, and `QueryExecutor`. + +### External +- YAML plus Jinja2 template syntax inside SQL. + + diff --git a/tests/AGENTS.md b/tests/AGENTS.md new file mode 100644 index 0000000..e40bde0 --- /dev/null +++ b/tests/AGENTS.md @@ -0,0 +1,53 @@ + + + +# tests + +## Purpose +`tests` contains the pytest suite for CLI commands, configuration and focus resolution, SQLite context validation, MCP server tools, public API behavior, domain models, and query subsystem components. Tests use temporary SQLite fixtures to avoid dependence on local snapshot files. + +## Key Files +| File | Description | +|------|-------------| +| `conftest.py` | Shared pytest fixtures, including temporary snapshot database setup. | +| `run_tests.sh` | Convenience script for running the test suite with coverage. | +| `test_api.py` | Tests for the public `SnapshotAnalyzer` API layer. | +| `test_cli.py` | CLI behavior tests, including focus, query listing, template info, and output limits. | +| `test_completion.py` | Shell completion helper tests. | +| `test_config.py` | Configuration and focus precedence tests. | +| `test_context.py` | SQLite context, schema validation, and device discovery tests. | +| `test_mcp_server.py` | MCP server tool/resource/prompt behavior tests. | +| `test_models.py` | Package-level model behavior tests. | +| `test_package.py` | Package metadata/import/version tests. | + +## Subdirectories +| Directory | Purpose | +|-----------|---------| +| `core/` | Service-layer tests for focus and query orchestration (see `core/AGENTS.md`). | +| `models/` | Domain model and enum tests (see `models/AGENTS.md`). | +| `query/` | Query builder, config, executor, mapper, registry, and condition tests (see `query/AGENTS.md`). | + +## For AI Agents + +### Working In This Directory +- Add or update focused tests with behavior changes; `tests/test_cli.py` is the first place to check for user-facing command behavior. +- Use temporary directories and SQLite fixtures instead of relying on local `.pt-snap/focus.json` or real snapshot databases. +- Keep tests deterministic across working directories and user machines. + +### Testing Requirements +- Run `conda run -n cc pytest` for the full suite when available. +- Use targeted commands such as `pytest tests/query/test_executor.py` for narrow query changes. + +### Common Patterns +- CLI tests exercise Typer commands through test runners and assert printed output. +- Query tests build minimal SQLite schemas and YAML-like config objects around the template pipeline. + +## Dependencies + +### Internal +- Tests import `pt_snap_cli` package modules from `src/` via editable install or test environment path configuration. + +### External +- `pytest` is the test runner. + + diff --git a/tests/core/AGENTS.md b/tests/core/AGENTS.md new file mode 100644 index 0000000..f239f8c --- /dev/null +++ b/tests/core/AGENTS.md @@ -0,0 +1,41 @@ + + + +# core tests + +## Purpose +`tests/core` verifies the service layer that sits between CLI/API/MCP callers and lower-level config, context, and query modules. + +## Key Files +| File | Description | +|------|-------------| +| `test_focus_service.py` | Tests focus service resolution, validation, project/global writes, and device handling. | +| `test_query_service.py` | Tests query service template listing/info, focus resolution, execution, row limiting, and error translation. | + +## Subdirectories +| Directory | Purpose | +|-----------|---------| +| None | Service tests are flat. | + +## For AI Agents + +### Working In This Directory +- Update these tests when `src/pt_snap_cli/core/` behavior changes. +- Assert service-level errors rather than low-level implementation exceptions. + +### Testing Requirements +- Run `pytest tests/core` for service-layer changes. + +### Common Patterns +- Tests isolate focus/query services from persistent user config and use temporary fixtures. + +## Dependencies + +### Internal +- `src/pt_snap_cli/core/` is the primary code under test. +- Shared fixtures may come from `tests/conftest.py`. + +### External +- `pytest`. + + diff --git a/tests/models/AGENTS.md b/tests/models/AGENTS.md new file mode 100644 index 0000000..eb5d0a3 --- /dev/null +++ b/tests/models/AGENTS.md @@ -0,0 +1,42 @@ + + + +# model tests + +## Purpose +`tests/models` verifies snapshot domain models and enum definitions exposed from `src/pt_snap_cli/models`. + +## Key Files +| File | Description | +|------|-------------| +| `__init__.py` | Test package marker. | +| `test_block.py` | Block model tests. | +| `test_enums.py` | Enum value and behavior tests. | +| `test_event.py` | Event model tests. | + +## Subdirectories +| Directory | Purpose | +|-----------|---------| +| None | Model tests are flat. | + +## For AI Agents + +### Working In This Directory +- Keep tests aligned with public model exports and expected snapshot semantics. +- Add enum coverage when introducing or renaming model enum values. + +### Testing Requirements +- Run `pytest tests/models` for model changes. + +### Common Patterns +- Tests focus on object construction, field behavior, and enum correctness. + +## Dependencies + +### Internal +- `src/pt_snap_cli/models/` is the primary code under test. + +### External +- `pytest`. + + diff --git a/tests/query/AGENTS.md b/tests/query/AGENTS.md new file mode 100644 index 0000000..d1319f1 --- /dev/null +++ b/tests/query/AGENTS.md @@ -0,0 +1,48 @@ + + + +# query tests + +## Purpose +`tests/query` verifies the query subsystem: fluent SQL building, condition composition, YAML config loading, Jinja2 rendering, SQLite execution, result mapping, and registry behavior. + +## Key Files +| File | Description | +|------|-------------| +| `__init__.py` | Test package marker. | +| `test_builder.py` | Tests `QueryBuilder` SQL and parameter generation. | +| `test_condition.py` | Tests condition objects and composite boolean conditions. | +| `test_config.py` | Tests YAML loading and query parameter validation. | +| `test_executor.py` | Tests template rendering and execution against SQLite fixtures. | +| `test_mapper.py` | Tests result type conversion and model factory mapping. | +| `test_registry.py` | Tests template registration, lookup, category listing, and registry reset behavior. | + +## Subdirectories +| Directory | Purpose | +|-----------|---------| +| None | Query tests are flat. | + +## For AI Agents + +### Working In This Directory +- Add coverage in the most specific test file for any query subsystem change. +- Use minimal SQLite fixtures for executor tests rather than external databases. + +### Testing Requirements +- Run `pytest tests/query` for query subsystem changes. +- Run CLI tests too when changes affect user-visible template listing, metadata, or output. + +### Common Patterns +- Registry tests reset singleton state to avoid cross-test contamination. +- Executor tests exercise rendered SQL and device-specific table names. + +## Dependencies + +### Internal +- `src/pt_snap_cli/query/` and packaged YAML templates are the primary code under test. +- Shared database fixtures may come from `tests/conftest.py`. + +### External +- `pytest`. + + From d4159ffad22408416d62f46a5a3997471f89d1d4 Mon Sep 17 00:00:00 2001 From: codeasier Date: Sat, 8 Aug 2026 09:22:16 +0800 Subject: [PATCH 2/3] docs: align guides with runtime behavior --- README.md | 72 ++-------------- README_zh.md | 59 ++----------- docs/AGENTS.md | 7 +- docs/README.md | 14 ++- docs/en/AGENTS.md | 6 +- docs/en/database.md | 103 +++++++++++++--------- docs/en/focus-management.md | 11 ++- docs/en/mcp.md | 8 +- docs/en/querying.md | 48 +++++++---- docs/en/quickstart.md | 20 +++-- docs/en/result-mapper-api.md | 4 +- docs/en/snapshot-analyzer-api.md | 143 +++++++++++++++++++++++++++++++ docs/en/splitting.md | 21 +++-- docs/zh/AGENTS.md | 6 +- docs/zh/database.md | 112 +++++++++++++++--------- docs/zh/focus-management.md | 9 +- docs/zh/mcp.md | 8 +- docs/zh/querying.md | 42 +++++---- docs/zh/quickstart.md | 18 ++-- docs/zh/result-mapper-api.md | 2 +- docs/zh/snapshot-analyzer-api.md | 130 ++++++++++++++++++++++++++++ docs/zh/splitting.md | 14 ++- 22 files changed, 582 insertions(+), 275 deletions(-) create mode 100644 docs/en/snapshot-analyzer-api.md create mode 100644 docs/zh/snapshot-analyzer-api.md diff --git a/README.md b/README.md index e9db587..2f8103b 100644 --- a/README.md +++ b/README.md @@ -6,6 +6,8 @@ A command-line tool for analyzing PyTorch memory snapshots. Set a snapshot datab ## Installation +From a source checkout: + ```bash pip install -e . ``` @@ -25,7 +27,7 @@ pt-snap query --list ```bash # Set the snapshot database and device -pt-snap focus examples/snapshot_expandable.pkl.db --device 0 +pt-snap focus snapshot.pkl.db --device 0 # List available queries pt-snap query --list @@ -71,21 +73,12 @@ See the [full quick start guide](docs/en/quickstart.md) for a walkthrough. pt-snap-mcp ``` -The server exposes the following tools: - -| Tool | Description | -|------|-------------| -| `get_focus` | Get the current analysis focus | -| `set_focus` | Set focus to a database and optional device | -| `list_templates` | List available query templates | -| `get_template_info` | Get template details and parameters | -| `execute_query` | Run a query template against the focused database | -| `get_database_metadata` | Inspect import metadata for the focused or specified database | - See the [MCP guide](docs/en/mcp.md) for setup and usage details. ## Documentation +See the [documentation index](docs/README.md) for all English and Chinese guides. + | Topic | Guide | |-------|-------| | Getting started | [Quick Start](docs/en/quickstart.md) | @@ -94,61 +87,14 @@ See the [MCP guide](docs/en/mcp.md) for setup and usage details. | Splitting snapshots | [Splitting Snapshots](docs/en/splitting.md) | | MCP server | [MCP Guide](docs/en/mcp.md) | | Database format | [SnapshotDB Schema](docs/en/database.md) | -| Python API | [ResultMapper API](docs/en/result-mapper-api.md) | - -## Query Templates - -9 built-in templates across 3 categories: - -- **Basic**: `block`, `event`, `allocation` -- **Statistical**: `active_blocks_at_event`, `allocator_gap`, `callstack_analysis`, `memory_peak` -- **Business**: `active_memory_callstack_at_event`, `leak_detection` - -See [Querying](docs/en/querying.md) for details. - -## Project Structure - -``` -pt-snap-cli/ -├── src/ -│ └── pt_snap_cli/ -│ ├── cli.py # CLI entry point -│ ├── context.py # Database context manager -│ ├── config.py # Focus management -│ ├── api.py # Python API layer -│ ├── query/ -│ │ ├── builder.py # Query builder -│ │ ├── executor.py # Query executor -│ │ ├── mapper.py # Result mapper -│ │ ├── registry.py # Query registry -│ │ ├── condition.py # Query conditions -│ │ ├── config.py # Query configuration -│ │ └── templates/ # Query templates -│ ├── models/ # Data models -│ └── mcp/ # MCP server for agent integration -├── tests/ # Test files -├── examples/ # Example data -└── docs/ # Documentation -``` +| Python API | [SnapshotAnalyzer API](docs/en/snapshot-analyzer-api.md) | +| Result mapping utility | [ResultMapper API](docs/en/result-mapper-api.md) | ## Development ```bash +pip install -e ".[dev]" # Install development dependencies pytest # Run all tests -./tests/run_tests.sh # Full test run with coverage -black . && ruff check . # Format and lint +black --check . && ruff check . # Check formatting and lint python -m build # Build sdist and wheel ``` - -### CLI/MCP semantic contract - -The CLI and MCP server must keep using the shared API/core services as the single -semantic source of truth. The CLI may render results as terminal text and the MCP -server may return JSON-like tool payloads, but normalized meanings must stay in -sync for focus handling, query template listing, template metadata, query -execution, and shared error cases. - -`tests/test_contract_cli_mcp.py` compares normalized CLI output with MCP tool -results on the same inputs and fixtures. When adding a new shared CLI/MCP capability, -update or add a contract case in that file so CI fails if either adapter drifts -from the shared behavior. diff --git a/README_zh.md b/README_zh.md index c8134ef..0c6dd0d 100644 --- a/README_zh.md +++ b/README_zh.md @@ -6,6 +6,8 @@ ## 安装 +从源码 checkout 安装: + ```bash pip install -e . ``` @@ -21,7 +23,7 @@ pt-snap import snapshot.pkl pt-snap metadata snapshot.pkl.db # 设置快照数据库和设备 -pt-snap focus examples/snapshot_expandable.pkl.db --device 0 +pt-snap focus snapshot.pkl.db --device 0 # 列出可用查询 pt-snap query --list @@ -67,21 +69,12 @@ pt-snap split snapshot.pkl --slices 4 --output snapshot-slices pt-snap-mcp ``` -服务器暴露以下工具: - -| 工具 | 说明 | -|------|------| -| `get_focus` | 获取当前分析焦点 | -| `set_focus` | 设置焦点到指定数据库和设备 | -| `list_templates` | 列出可用的查询模板 | -| `get_template_info` | 获取模板详情和参数 | -| `execute_query` | 对焦点数据库执行查询模板 | -| `get_database_metadata` | 查看焦点或指定数据库的导入 metadata | - 详见 [MCP 指南](docs/zh/mcp.md)。 ## 文档 +全部中英文指南见[文档索引](docs/README.md)。 + | 主题 | 指南 | |------|------| | 入门指南 | [Quick Start](docs/zh/quickstart.md) | @@ -90,48 +83,14 @@ pt-snap-mcp | 拆分快照 | [拆分快照](docs/zh/splitting.md) | | MCP 服务器 | [MCP 指南](docs/zh/mcp.md) | | 数据库格式 | [SnapshotDB Schema](docs/zh/database.md) | -| Python API | [ResultMapper API](docs/zh/result-mapper-api.md) | - -## 查询模板 - -9 个内置模板,3 个分类: - -- **Basic**: `block`, `event`, `allocation` -- **Statistical**: `active_blocks_at_event`, `allocator_gap`, `callstack_analysis`, `memory_peak` -- **Business**: `active_memory_callstack_at_event`, `leak_detection` - -详见 [Querying](docs/zh/querying.md)。 - -## 项目结构 - -``` -pt-snap-cli/ -├── src/ -│ └── pt_snap_cli/ -│ ├── cli.py # CLI 入口 -│ ├── context.py # 数据库连接管理器 -│ ├── config.py # 焦点管理 -│ ├── api.py # Python API 层 -│ ├── query/ -│ │ ├── builder.py # 查询构建器 -│ │ ├── executor.py # 查询执行器 -│ │ ├── mapper.py # 结果映射器 -│ │ ├── registry.py # 查询注册表 -│ │ ├── condition.py # 查询条件 -│ │ ├── config.py # 查询配置 -│ │ └── templates/ # 查询模板 -│ ├── models/ # 数据模型 -│ └── mcp/ # MCP 服务器(Agent 集成) -├── tests/ # 测试文件 -├── examples/ # 示例数据 -└── docs/ # 文档 -``` +| Python API | [SnapshotAnalyzer API](docs/zh/snapshot-analyzer-api.md) | +| 结果映射工具 | [ResultMapper API](docs/zh/result-mapper-api.md) | ## 开发 ```bash +pip install -e ".[dev]" # 安装开发依赖 pytest # 运行所有测试 -./tests/run_tests.sh # 完整测试(含覆盖率) -black . && ruff check . # 格式化和 lint +black --check . && ruff check . # 检查格式和 lint python -m build # 构建 sdist 和 wheel ``` diff --git a/docs/AGENTS.md b/docs/AGENTS.md index 20214f6..498bb61 100644 --- a/docs/AGENTS.md +++ b/docs/AGENTS.md @@ -1,10 +1,10 @@ - + # docs ## Purpose -`docs` contains end-user and API documentation for the CLI, focus management, query templates, MCP integration, snapshot database format, and result mapping API. It is split into English and Chinese language trees with a top-level README for navigation. +`docs` contains end-user and API documentation for the CLI, focus management, query templates, MCP integration, snapshot database format, high-level and result mapping Python APIs, and retained legal evidence. It is split into English and Chinese language trees with a top-level README for navigation. ## Key Files | File | Description | @@ -16,6 +16,7 @@ |-----------|---------| | `en/` | English documentation guides (see `en/AGENTS.md`). | | `zh/` | Chinese documentation guides (see `zh/AGENTS.md`). | +| `legal/` | Governance evidence referenced by the documentation index and snapshot provenance record. | ## For AI Agents @@ -29,7 +30,7 @@ - If examples depend on behavior changes, run the matching CLI or service tests. ### Common Patterns -- The two language trees mirror the same topics: quick start, focus management, querying, MCP, database schema, and result mapper API. +- The two language trees mirror the same topics: quick start, focus management, querying, snapshot splitting, MCP, database schema, SnapshotAnalyzer API, and result mapper API. ## Dependencies diff --git a/docs/README.md b/docs/README.md index d7c4a52..69fba51 100644 --- a/docs/README.md +++ b/docs/README.md @@ -5,11 +5,15 @@ | Topic | File | |-------|------| | Quick Start | [en/quickstart.md](en/quickstart.md) | -| Context Management | [en/focus-management.md](en/focus-management.md) | +| Importing snapshots | [en/quickstart.md](en/quickstart.md) | +| Focus Management | [en/focus-management.md](en/focus-management.md) | | Querying | [en/querying.md](en/querying.md) | +| Reports | [en/querying.md](en/querying.md) | | Splitting Snapshots | [en/splitting.md](en/splitting.md) | | MCP server | [en/mcp.md](en/mcp.md) | | Database Schema | [en/database.md](en/database.md) | +| Import metadata | [en/database.md](en/database.md) | +| SnapshotAnalyzer API | [en/snapshot-analyzer-api.md](en/snapshot-analyzer-api.md) | | ResultMapper API | [en/result-mapper-api.md](en/result-mapper-api.md) | ## 中文 @@ -17,9 +21,17 @@ | 主题 | 文件 | |------|------| | 快速开始 | [zh/quickstart.md](zh/quickstart.md) | +| 导入快照 | [zh/quickstart.md](zh/quickstart.md) | | Focus 管理 | [zh/focus-management.md](zh/focus-management.md) | | 运行查询 | [zh/querying.md](zh/querying.md) | +| 分析报告 | [zh/querying.md](zh/querying.md) | | 拆分快照 | [zh/splitting.md](zh/splitting.md) | | MCP 服务器 | [zh/mcp.md](zh/mcp.md) | | 数据库格式 | [zh/database.md](zh/database.md) | +| 导入 metadata | [zh/database.md](zh/database.md) | +| SnapshotAnalyzer API | [zh/snapshot-analyzer-api.md](zh/snapshot-analyzer-api.md) | | ResultMapper API | [zh/result-mapper-api.md](zh/result-mapper-api.md) | + +## Governance + +- [Snapshot runtime MIT relicensing evidence](legal/mem%73napdump-mit-relicensing.md) diff --git a/docs/en/AGENTS.md b/docs/en/AGENTS.md index 3dec0d4..a18e379 100644 --- a/docs/en/AGENTS.md +++ b/docs/en/AGENTS.md @@ -1,10 +1,10 @@ - + # en ## Purpose -`docs/en` contains the English documentation set for installing and using `pt-snap-cli`, managing focus, querying snapshot databases, running the MCP server, understanding the SQLite schema, and using result mapping APIs. +`docs/en` contains the English documentation set for installing and using `pt-snap-cli`, managing focus, querying and splitting snapshots, running the MCP server, understanding the SQLite schema, and using the high-level and result mapping Python APIs. ## Key Files | File | Description | @@ -12,8 +12,10 @@ | `quickstart.md` | End-to-end installation and first-query walkthrough. | | `focus-management.md` | Focus resolution, project focus files, environment variables, and global config behavior. | | `querying.md` | Query listing, template info, parameter usage, result limits, and built-in templates. | +| `splitting.md` | Snapshot slicing strategies, output formats, replay validation, and publication guarantees. | | `mcp.md` | MCP server setup, tools, resources, prompts, and agent usage. | | `database.md` | Snapshot SQLite table/schema reference. | +| `snapshot-analyzer-api.md` | High-level Python API for focus, template discovery, queries, and import metadata. | | `result-mapper-api.md` | Result mapping API documentation. | ## Subdirectories diff --git a/docs/en/database.md b/docs/en/database.md index bef6ba4..9aea644 100644 --- a/docs/en/database.md +++ b/docs/en/database.md @@ -1,6 +1,6 @@ # SnapshotDB Schema -[English](database.md) | [中文](../zh/database.md) +[中文](../zh/database.md) | English ## Overview @@ -8,7 +8,7 @@ SnapshotDB is the SQLite database format for persisting PyTorch memory profiling 1. The total memory pool size and total allocated block size after any event; 2. The complete lifecycle of all memory blocks during the collection period (which event allocated them, which event freed them). -**Example database file**: `snapshot_expandable.pkl.db` +**Example database file**: `snapshot.pkl.db` --- @@ -59,7 +59,7 @@ optional device; query templates then resolve those device-specific table names: ```bash pt-snap focus snapshot.pkl.db --device 0 pt-snap query --template-use memory_peak -pt-snap query --template-use block --params '{"state": 1}' +pt-snap query --template-use block --params '{"min_size": 1048576}' ``` Use `pt-snap query --list` and `pt-snap query --template-info ` to inspect @@ -76,6 +76,7 @@ the supported query surface. See [Querying](querying.md) for the complete workfl | `dictionary` | Enum mapping dictionary | - | | `trace_entry_0` | Device 0 event trace table | 8,094 | | `block_0` | Device 0 memory block table | - | +| `pt_snap_metadata` | First-party import provenance and cache metadata | 1 | > **Naming convention**: For multi-device scenarios, table names are suffixed with the device ID, e.g., `trace_entry_1` and `block_1` for device 1. @@ -92,13 +93,13 @@ Records complete trace information for memory management events. ```sql CREATE TABLE trace_entry_0 ( `id` INTEGER PRIMARY KEY, - `action` INTEGER NOT NULL, - `address` INTEGER NOT NULL, - `size` INTEGER NOT NULL, - `stream` INTEGER NOT NULL, - `allocated` INTEGER NOT NULL, - `active` INTEGER NOT NULL, - `reserved` INTEGER NOT NULL, + `action` INTEGER, + `address` INTEGER, + `size` INTEGER, + `stream` INTEGER, + `allocated` INTEGER, + `active` INTEGER, + `reserved` INTEGER, `callstack` TEXT ); ``` @@ -108,13 +109,13 @@ CREATE TABLE trace_entry_0 ( | Column | Type | Constraints | Description | Example | |--------|------|-------------|-------------|---------| | id | INTEGER | PRIMARY KEY | Unique event ID; negative values indicate system-generated events | `1`, `-100` | -| action | INTEGER | NOT NULL | Action type code (see below) | `4` | -| address | INTEGER | NOT NULL | Memory address | `20697535234048` | -| size | INTEGER | NOT NULL | Allocation size in bytes | `41943040` | -| stream | INTEGER | NOT NULL | CUDA/CANN stream ID | `1276474240` | -| allocated | INTEGER | NOT NULL | Total allocated bytes | `136426496` | -| active | INTEGER | NOT NULL | Total active bytes | `136426496` | -| reserved | INTEGER | NOT NULL | Memory pool reserved bytes | `155189248` | +| action | INTEGER | — | Action type code (see below) | `4` | +| address | INTEGER | — | Memory address | `20697535234048` | +| size | INTEGER | — | Allocation size in bytes | `41943040` | +| stream | INTEGER | — | CUDA/CANN stream ID | `1276474240` | +| allocated | INTEGER | — | Total allocated bytes | `136426496` | +| active | INTEGER | — | Total active bytes | `136426496` | +| reserved | INTEGER | — | Memory pool reserved bytes | `155189248` | | callstack | TEXT | NULL | Callstack information (multi-line text) | See example | #### Action Type Codes @@ -160,26 +161,26 @@ Records detailed information about memory blocks and their lifecycle state. ```sql CREATE TABLE block_0 ( `id` INTEGER PRIMARY KEY, - `address` INTEGER NOT NULL, - `size` INTEGER NOT NULL, - `requestedSize` INTEGER NOT NULL DEFAULT 99, - `state` INTEGER NOT NULL DEFAULT 99, - `allocEventId` INTEGER DEFAULT -1, - `freeEventId` INTEGER DEFAULT -1 + `address` INTEGER, + `size` INTEGER, + `requestedSize` INTEGER, + `state` INTEGER DEFAULT 99, + `allocEventId` INTEGER, + `freeEventId` INTEGER ); ``` #### Columns -| Column | Type | Default | Description | +| Column | Type | SQL default | Description | |--------|------|---------|-------------| -| id | INTEGER | — | Unique block ID, typically negative | +| id | INTEGER | — | Unique block ID; dynamic blocks use their allocation event ID, while preexisting blocks use a negative ID | | address | INTEGER | — | Memory address | | size | INTEGER | — | Actual allocated size (includes alignment overhead) | -| requestedSize | INTEGER | 99 | User-requested allocation size | +| requestedSize | INTEGER | — | User-requested allocation size | | state | INTEGER | 99 | State code (see below) | -| allocEventId | INTEGER | -1 | Associated allocation event ID | -| freeEventId | INTEGER | -1 | Associated free event ID | +| allocEventId | INTEGER | — | Associated allocation event ID; the producer writes `-1` when it was not captured | +| freeEventId | INTEGER | — | Associated free-completion event ID; the producer writes `-1` when it was not captured | #### State Codes @@ -208,7 +209,7 @@ CREATE TABLE block_0 ( | Condition | Constraint | |-----------|------------| -| `freeEventId >= 0` | `block.id` matches `freeEventId`, pointing to the same-ID free event in `trace_entry` | +| `freeEventId >= 0` | Points to the corresponding free-completion event in `trace_entry`; it is independent of `block.id` | | `freeEventId == -1` | The free completion event for this block was not captured during snapshot collection (the block was not freed when collection ended) | #### State Usage @@ -217,11 +218,10 @@ CREATE TABLE block_0 ( #### Size Calculation -`block.size` is the actual allocated size (including alignment overhead), while `block.requestedSize` is the user-requested size. The calculation is: - -``` -size = math.ceil((requestedSize + 32) / 512) * 512 -``` +`block.size` is the actual allocated size, while `block.requestedSize` is the +user-requested size. Both values come from the snapshot producer or replayed +allocator state. Their relationship depends on allocator configuration and must +not be inferred with one universal alignment formula. #### Data Example @@ -242,8 +242,8 @@ Stores enum value mappings used to decode fields like `action` and `state`. CREATE TABLE dictionary ( `table` TEXT, -- Table name `column` TEXT, -- Column name - `key` TEXT, -- Original string value - `value` TEXT -- Encoded integer value (as string) + `key` TEXT, -- Encoded integer value (as string) + `value` TEXT -- Original string value ); ``` @@ -253,12 +253,35 @@ CREATE TABLE dictionary ( |--------|------|-------------| | table | TEXT | Parent table name, e.g., `trace_entry_0`, `block_0` | | column | TEXT | Column name, e.g., `action`, `state` | -| key | TEXT | Original string value, e.g., `alloc`, `active_allocated` | -| value | TEXT | Encoded integer value as string, e.g., `4`, `1` | +| key | TEXT | Encoded integer value as string, e.g., `4`, `1` | +| value | TEXT | Original string value, e.g., `alloc`, `active_allocated` | #### Data Example ``` -table=trace_entry_0, column=action, key=alloc, value=4 -table=block_0, column=state, key=active_allocated, value=1 +table=trace_entry_0, column=action, key=4, value=alloc +table=block_0, column=state, key=1, value=active_allocated +``` + +### 4. pt_snap_metadata — Import Metadata Table + +First-party imports create one metadata row for provenance and cache +compatibility checks: + +```sql +CREATE TABLE pt_snap_metadata ( + id INTEGER PRIMARY KEY CHECK (id = 1), + metadata_schema_version INTEGER NOT NULL, + import_format_version INTEGER NOT NULL, + source_sha256 TEXT NOT NULL, + source_size INTEGER NOT NULL, + source_name TEXT NOT NULL, + requested_device INTEGER, + importer_name TEXT NOT NULL, + importer_version TEXT NOT NULL, + completed_at TEXT NOT NULL +); ``` + +Legacy and externally produced compatible databases may omit this table. They +remain queryable, but `pt-snap metadata` reports their metadata as unavailable. diff --git a/docs/en/focus-management.md b/docs/en/focus-management.md index 1ac9be2..3a36614 100644 --- a/docs/en/focus-management.md +++ b/docs/en/focus-management.md @@ -2,7 +2,10 @@ [中文](../zh/focus-management.md) | English -`pt-snap` supports project-scoped focus so different terminals, agents, or working directories can analyze different databases and devices without interfering with each other. +`pt-snap` supports project-scoped focus. Processes that resolve the same nearest +`.pt-snap/focus.json` share that focus, while different project directories can +select different databases and devices. Use a session override when terminals or +agents in the same project must remain isolated. ## Resolution Priority @@ -84,9 +87,13 @@ pt-snap config --clear # Clear global configuration | Scope | File | |-------|------| -| Project | `.pt-snap/focus.json` (git-ignored, per-project) | +| Project | `.pt-snap/focus.json` (shared by processes in the project scope) | | Global | `~/.config/pt-snap-cli/config.json` | +Project focus can contain an absolute local database path. Add `.pt-snap/` to the +project's `.gitignore` when the file should remain local; pt-snap does not modify +ignore rules automatically. + ### Focus File Format ```json diff --git a/docs/en/mcp.md b/docs/en/mcp.md index 73b5b2a..a63388d 100644 --- a/docs/en/mcp.md +++ b/docs/en/mcp.md @@ -6,6 +6,8 @@ ## Installation +From a source checkout: + ```bash pip install -e . ``` @@ -45,7 +47,7 @@ This starts a FastMCP server that exposes tools for analyzing PyTorch memory sna ## Typical Workflow -1. Call `set_focus` with `db_path` to point to a snapshot file +1. Call `set_focus` with `db_path` to point to a SnapshotDB `.db` file 2. Call `list_templates` to discover available queries 3. Call `get_template_info` with a template name to see its parameters 4. Call `execute_query` with the template name and parameters @@ -54,7 +56,7 @@ This starts a FastMCP server that exposes tools for analyzing PyTorch memory sna ## Example Usage ```python -# Set focus to a snapshot +# Set focus to a SnapshotDB file set_focus(db_path="/path/to/snapshot.db", device_id=0) # List templates @@ -63,7 +65,7 @@ list_templates() # Get template details get_template_info("leak_detection") -# Returns: {"name": "leak_detection", "parameters": [...], ...} +# Returns: {"name": "leak_detection", "parameters": {"min_size": {...}}, ...} # Run a query execute_query("leak_detection", params={"min_size": 1024}) diff --git a/docs/en/querying.md b/docs/en/querying.md index 7d1c0fe..5183bd5 100644 --- a/docs/en/querying.md +++ b/docs/en/querying.md @@ -1,26 +1,29 @@ # Querying -[English](querying.md) | [中文](../zh/querying.md) +[中文](../zh/querying.md) | English Run memory analysis queries against your snapshot database. ## The Query Command ```bash -pt-snap query [--template-use ] [--params ] [--device ] [--list] [--template-info