Skip to content

feat: add peak memory attribution reports - #63

Merged
codeasier merged 4 commits into
mainfrom
worktree-issue-62-peak-memory-report
Jun 7, 2026
Merged

codeasier merged 4 commits into
mainfrom
worktree-issue-62-peak-memory-report

Conversation

@codeasier

@codeasier codeasier commented Jun 7, 2026 •

Copy link
Copy Markdown
Owner

📝 Description

This PR adds a user-facing peak memory attribution workflow to pt-snap, so users can move from "find the peak" to "explain what was live at that moment" without writing manual SQL or Python scripts.

User-visible changes in this PR:

  1. New Statistical Query: active_blocks_at_event

    • Lists blocks that are still live at a given event.
    • Supports optional static memory inclusion.
    • Live block rule:
      • allocEventId <= event_id
      • and freeEventId = -1 or freeEventId > event_id
    • Static memory rule:
      • allocEventId = -1 AND freeEventId = -1
  2. New Statistical Query: allocator_gap

    • Compares the peak events for allocated, active, and reserved.
    • Reports whether those peaks happen at the same event.
    • Reports same-event gaps such as reserved - active and reserved - allocated.
    • Helps avoid misinterpreting peak values from different events as a same-moment fragmentation/cache gap.
  3. New Business Query: active_memory_callstack_at_event

    • Starts from the active block set at an event.
    • Joins dynamic blocks back to their allocation events.
    • Aggregates active memory by allocation callstack.
    • Separates static memory into its own group instead of inventing a callstack.
  4. New CLI report command: pt-snap report peak-memory

    • Provides a higher-level summary combining:
      • memory_peak
      • allocator_gap
      • active_memory_callstack_at_event
    • Supports:
      • --metric active|allocated|reserved
      • --include-static|--exclude-static
      • --limit
      • --json
  5. Packaging fix

    • Categorized YAML query templates are now included in wheel/sdist artifacts.

Basic usage examples now supported:

# Find peak events
pt-snap query --template-use memory_peak

# Inspect blocks live at a chosen event
pt-snap query --template-use active_blocks_at_event --params '{"event_id":1234,"include_static":true}'

# Attribute live memory at that event to callstacks
pt-snap query --template-use active_memory_callstack_at_event --params '{"event_id":1234,"include_static":true,"top_n":20}'

# Compare allocated/active/reserved peak relationships
pt-snap query --template-use allocator_gap

# Generate a higher-level report
pt-snap report peak-memory /path/to/snapshot.db --metric active
pt-snap report peak-memory /path/to/snapshot.db --json

🔗 Related Issue

Fixes #62

🧪 Testing

  • Tests added/updated
  • Manual testing completed
  • conda run -n cc pytest
  • conda run -n cc python -m build
  • conda run -n cc ruff check .
  • conda run -n cc black --check .
  • conda run -n cc python -m basedpyright --pythonpath "$(conda run -n cc python -c 'import sys; print(sys.executable)')"

✅ Checklist

  • Code follows project guidelines
  • Documentation updated (docs/en/querying.md, docs/zh/querying.md)
  • No new warnings introduced
  • Changes tested locally

📸 Screenshots (if applicable)

N/A

📋 Additional Notes

  • This PR now includes docs for both query-template usage and the new pt-snap report peak-memory workflow.
  • CI follow-up fixes were pushed in-place to the same PR branch; the PR was not closed or recreated.

codeasier added 2 commits June 7, 2026 13:09
Add event-scoped active block and callstack attribution queries plus an initial peak-memory report command so users can inspect peak composition without manual SQL scripts. Also package categorized query templates in built artifacts and cover the new workflow with service, template, and CLI tests.
Update brittle _safe_call tests to mock the CLI entrypoint instead of Click internals, align invalid metric expectations with Typer's argument validation, and format the new template tests so CI lint and type checks pass consistently across environments.

@codeasier codeasier left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review

Found 2 issues.

Comment thread src/pt_snap_cli/cli.py Outdated
)
raise typer.Exit(1) from None
except DatabaseMissingError:
typer.secho(f"Error: Database not found: {db_path}", fg=typer.colors.RED)

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When report peak-memory is run without an explicit db_path, a stale focused/configured database will reach this handler with db_path == None, so the user sees Database not found: None. The existing query flow re-resolves focus and reports the focused source/path; this new command should surface the resolved missing path too so users can fix the stale focus.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in the report CLI handler. It now re-resolves focus on DatabaseMissingError, reports the actual focused source/path, and includes the focus file when available instead of printing Database not found: None.

WHERE size >= {{ min_size|int }}
AND (
{% if include_static %}(allocEventId = -1 AND freeEventId = -1) OR{% endif %}
(allocEventId != -1 AND allocEventId <= {{ event_id|int }} AND (freeEventId = -1 OR freeEventId > {{ event_id|int }}))

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This active-block predicate omits live blocks encoded with freeEventId IS NULL. Existing templates treat freeEventId IS NULL OR freeEventId < 0 as not freed, so snapshots using NULL for open allocations will undercount peak active memory and misattribute callstack totals. The sibling statistical/active_blocks_at_event.yaml uses the same predicate and likely needs the same fix.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in both active-block templates. The live-block predicate now treats freeEventId IS NULL and other negative values as not freed, matching the existing repository semantics used by leak_detection. Added targeted tests for both active_blocks_at_event and active_memory_callstack_at_event.

codeasier added 2 commits June 7, 2026 18:34
Document the new event-scoped peak attribution queries and the peak-memory report command, including the lifecycle rules behind active blocks, static memory handling, allocator gap interpretation, and example CLI usage in both English and Chinese docs.
Make report peak-memory surface the resolved missing database path for stale focused configurations, and treat freeEventId NULL or negative values as not-freed in the new active block attribution queries. Cover both cases with focused CLI and query-template tests.
@codeasier
codeasier merged commit 2bb51a8 into main Jun 7, 2026
5 checks passed
@codeasier
codeasier deleted the worktree-issue-62-peak-memory-report branch June 7, 2026 11:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] 增加峰值内存拆解与报告能力

1 participant