Repository navigation
Conversation
added 3 commits
June 15, 2026 06:53
…a MemSnapDump Integrate codeasier/MemSnapDump's dump2db capability so users can convert a raw PyTorch memory snapshot (.pkl/.pickle) into a pt-snap-cli analyzable SQLite database without leaving the CLI. The generated .db is auto-persisted as project focus, so follow-up query/report commands work without repeating the path. Entry point (source-integrated, not PyPI): run_dump_to_db from memsnapdump.tools.adaptors.snapshot2db — note the actual module path differs from the issue's reference (memsnapdump.tools.dump2db), which only exposes main(). memsnapdump is declared as an optional extra pinned to the upstream main branch via git+https, since the project has no PyPI release or tags. New surface: - pt-snap import SNAPSHOT_FILE [--output-dir/-o] [--device/-d] [--no-focus] - core.ImportService with lazy import + single private call point (_run_dump_to_db) for testability; no subprocess fallback - ImportOptions / ImportResult frozen dataclasses - ImportToolMissingError / SnapshotFileInvalidError / ImportExecutionError Schema is fully compatible (MemSnapDump emits camelCase column names requestedSize/allocEventId/freeEventId which the existing query templates already use), so no Context / template / schema changes beyond a small discover_devices() convenience alias. Tests: 22 new (models, errors, ImportService incl. real-fixture main path, CLI command). Real .pkl fixtures from MemSnapDump's test_data are committed under tests/fixtures/snapshots/. 314 total passing, ruff/black/basedpyright clean. Out of scope per issue #65: split/clipping, API/MCP mirror, subprocess fallback. Closes #65
… raise The test workflow only installed .[dev], so the 8 tests exercising the real dump2db path failed with ImportToolMissingError in CI. Install .[dev,memsnapdump] so the real-fixture main-path tests run. basedpyright treats warnings as failures in this project. The import command had a redundant 'raise typer.Exit(code=1) from None' after _error(str(e)) — _error is NoReturn, making the raise unreachable. Drop it to match the existing focus/query command pattern.
The upstream codeasier/MemSnapDump repo uses 'master' as its default branch, not 'main'. The @main ref failed with 'pathspec main did not match' during pip install in CI.
codeasier
commented
Jun 15, 2026
| focus_state = self._focus_service.set_project_focus( | ||
| db_path=db_path, | ||
| device_id=options.device, | ||
| base_dir=output_dir, |
Owner
Author
There was a problem hiding this comment.
这里把 base_dir 设为 output_dir,会导致 focus 写到输出 DB 所在目录,而不是用户当前项目目录。比如在项目目录执行 pt-snap import /tmp/snapshot.pkl 时,后续从项目目录运行 pt-snap query 可能找不到刚写入的 focus,这和 PR 描述里“后续无需重复指定路径”的目标不一致。建议让 CLI 显式传入当前工作目录作为 project base,或者调整 set_project_focus 的默认语义。
codeasier
commented
Jun 15, 2026
| "basedpyright>=1.39.3", | ||
| ] | ||
| memsnapdump = [ | ||
| "memsnapdump @ git+https://github.com/codeasier/MemSnapDump.git@master", |
Owner
Author
There was a problem hiding this comment.
这里用的是 @master,但 PR 描述和文档里写的是 @main。如果上游默认分支已经是 main,这个 optional dependency 会直接安装失败。建议统一 pyproject.toml、文档和 PR 描述中的分支名。
13 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📝 Description
集成
codeasier/MemSnapDump的dump2db能力,新增pt-snap import子命令,将原始 PyTorch memory snapshot(.pkl/.pickle)一键转换为pt-snap-cli已支持分析的 SQLite.db,并自动写入项目级 focus,使后续pt-snap query/pt-snap report无需重复指定路径。实现要点:
memsnapdump.tools.adaptors.snapshot2db.run_dump_to_db(issue 提到的memsnapdump.tools.dump2db仅暴露main(),无法直接调用)。memsnapdump声明为 optional extra(memsnapdump @ git+https://github.com/codeasier/MemSnapDump.git@main),源码集成,缺包时给出清晰安装提示,不做 subprocess fallback。requestedSize/allocEventId/freeEventId)与现有 query 模板一致,无需新增 schema / 模板 / 修改 Context 核心逻辑(仅补一个discover_devices()便利别名)。ImportService采用 lazy import + 单一私有调用点_run_dump_to_db(便于 monkeypatch 测试)。🔗 Related Issue
Fixes #65
🧪 Testing
pt-snap import snapshot.pkl→ 生成 11MB db →pt-snap focus显示新 db →pt-snap query --template-use block返回 3219 行测试覆盖:
tests/core/test_import_models.py(4):ImportOptions/ImportResult 字段与冻结性tests/core/test_import_errors.py(3):3 个错误类型继承 PtSnapCoreErrortests/core/test_import_service.py(10):主路径(真实 fixture → db → focus)、多设备、5 类错误条件、--no-focus、.pickle后缀输出命名tests/test_cli.py::TestImportCommand(5):CLI 成功 / 缺包提示 / 非法路径 / 非法后缀 /--no-focusFixture:从 MemSnapDump
tests/test_data/引入 2 个真实.pkl(603K + 1.3M,合计 1.9M)到tests/fixtures/snapshots/,作为集成测试主路径。验证结果:
pytest:314 passed(292 基线 + 22 新增,0 失败)ruff check .:All checks passedblack --check .:57 files unchangedbasedpyright:0 errors(仅 memsnapdump 无类型标注导致的 reportAny 警告)✅ Checklist
📋 Additional Notes
与 issue #65 验收标准对照:
pyproject.toml新增memsnapdumpoptional extra,不加入默认 runtime dependenciespt-snap import SNAPSHOT_FILE,支持--output-dir/--device/--no-focus.pkl/.pickle输入生成<snapshot_file.name>.db.pt-snap/focus.json,后续pt-snap query/pt-snap report可复用memsnapdump时返回清晰错误与安装提示.pklfixture 覆盖主路径测试pt-snap split/ clipping,不新增 API/MCP 镜像,不新增 subprocess fallback入口路径纠正:issue 参考资源里写的
memsnapdump.tools.dump2db.run_dump_to_db实际只有main(),真正可调用的是memsnapdump.tools.adaptors.snapshot2db.run_dump_to_db,本 PR 已用正确路径。安装方式:MemSnapDump 不在 PyPI、无 git tag,故 optional extra 用
git+https://...@main锁定最新主分支;本地开发可用pip install -e /path/to/MemSnapDumpeditable 安装。后续 PR(out of scope):
pt-snap split时序切片、API/MCP 镜像、subprocess fallback。