Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 84 additions & 0 deletions docs/en/querying.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ Aggregation and analysis.
|----------|-------------|
| `callstack_analysis` | Analyze callstack information |
| `memory_peak` | Peak memory metrics |
| `active_blocks_at_event` | List blocks that are still active at a specific event, with optional static block inclusion |
| `allocator_gap` | Compare allocated, active, and reserved peak events and same-event gaps |

### Business Queries

Expand All @@ -52,6 +54,88 @@ Domain-specific analysis.
| Template | Description |
|----------|-------------|
| `leak_detection` | Find allocations without matching free events |
| `active_memory_callstack_at_event` | Aggregate blocks active at a specific event by allocation callstack, with static memory classified separately |

## Peak Memory Attribution Workflow

These additions productize the common "find the peak, then explain what was live at that moment" workflow.

### 1. Find the peak event

```bash
pt-snap query --template-use memory_peak
```

This returns peak values and the corresponding event IDs for `allocated`, `active`, and `reserved`.

### 2. Inspect the blocks that were still active at that event

```bash
pt-snap query --template-use active_blocks_at_event --params '{"event_id": 1234, "include_static": true}'
```

`active_blocks_at_event` treats a block as live at `event_id` when:

- `allocEventId <= event_id`
- and `freeEventId = -1` or `freeEventId > event_id`

When `include_static=true`, blocks with `allocEventId=-1 AND freeEventId=-1` are also included and labeled as `static`.

### 3. Attribute active memory to callstacks at that event

```bash
pt-snap query --template-use active_memory_callstack_at_event --params '{"event_id": 1234, "include_static": true, "top_n": 20}'
```

This query:

- starts from the active block set at `event_id`
- joins dynamic blocks back to their allocation events in `trace_entry_<device>`
- groups by allocation callstack
- emits static memory as a dedicated group instead of inventing a callstack

### 4. Compare peak event gaps across metrics

```bash
pt-snap query --template-use allocator_gap
```

This reports:

- the peak event for `allocated`, `active`, and `reserved`
- whether those peaks happen at the same event
- same-event gaps such as `reserved - active` and `reserved - allocated`

This is useful because `reserved` may peak at a different event from `active` or `allocated`, so subtracting peak values directly can be misleading.

## Report Command

For a higher-level summary, use the report command:

```bash
pt-snap report peak-memory [db_path] [--device <id>] [--metric active|allocated|reserved] [--include-static|--exclude-static] [--limit <n>] [--json]
```

Examples:

```bash
# Text report using the active peak event
pt-snap report peak-memory /path/to/snapshot.db

# Report the reserved peak instead
pt-snap report peak-memory /path/to/snapshot.db --metric reserved

# Emit machine-readable JSON
pt-snap report peak-memory /path/to/snapshot.db --json
```

The report command combines:

- `memory_peak`
- `allocator_gap`
- `active_memory_callstack_at_event`

and prints either a human-readable summary or JSON.

**Example:**
```bash
Expand Down
84 changes: 84 additions & 0 deletions docs/zh/querying.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ pt-snap query [--template-use <template_name>] [--params <json>] [--device <id>]
|------|------|
| `callstack_analysis` | 调用栈分析 |
| `memory_peak` | 峰值内存指标 |
| `active_blocks_at_event` | 查询某个事件时刻仍然活跃的 block,可选包含静态内存 |
| `allocator_gap` | 比较 allocated、active、reserved 三类峰值事件及其同事件 gap |

### Business Queries

Expand All @@ -52,6 +54,88 @@ pt-snap query [--template-use <template_name>] [--params <json>] [--device <id>]
| 模板 | 说明 |
|------|------|
| `leak_detection` | 查找未匹配释放事件的分配 |
| `active_memory_callstack_at_event` | 对某个事件时刻的活跃内存块按分配调用栈做聚合,并单独标识静态内存 |

## 峰值内存归因工作流

这些新增能力把“先找到峰值,再解释峰值时刻哪些内存仍然活跃”的手工分析流程产品化了。

### 1. 先定位峰值事件

```bash
pt-snap query --template-use memory_peak
```

这个模板会返回 `allocated`、`active`、`reserved` 的峰值,以及各自对应的事件 ID。

### 2. 查看该事件时刻仍然活跃的 block

```bash
pt-snap query --template-use active_blocks_at_event --params '{"event_id": 1234, "include_static": true}'
```

`active_blocks_at_event` 认为 block 在 `event_id` 时刻仍然活跃的条件是:

- `allocEventId <= event_id`
- 且 `freeEventId = -1` 或 `freeEventId > event_id`

当 `include_static=true` 时,还会包含 `allocEventId=-1 AND freeEventId=-1` 的 block,并标记为 `static`。

### 3. 对该时刻的活跃内存做调用栈归因

```bash
pt-snap query --template-use active_memory_callstack_at_event --params '{"event_id": 1234, "include_static": true, "top_n": 20}'
```

这个查询会:

- 先从 `event_id` 时刻的活跃 block 集合开始
- 把动态 block 回连到 `trace_entry_<device>` 的分配事件
- 按分配调用栈做聚合
- 对静态内存单独分组,而不是伪造调用栈

### 4. 比较不同指标的峰值与 gap

```bash
pt-snap query --template-use allocator_gap
```

这个模板会报告:

- `allocated`、`active`、`reserved` 的峰值事件
- 它们是否发生在同一个事件
- 同一事件上的 gap,例如 `reserved - active`、`reserved - allocated`

这样可以避免误把不同事件上的峰值直接相减,并错误解释为同一时刻的碎片或缓存 gap。

## Report 命令

如果需要更高层的摘要,可以使用:

```bash
pt-snap report peak-memory [db_path] [--device <id>] [--metric active|allocated|reserved] [--include-static|--exclude-static] [--limit <n>] [--json]
```

示例:

```bash
# 生成基于 active 峰值事件的文本报告
pt-snap report peak-memory /path/to/snapshot.db

# 改为查看 reserved 峰值
pt-snap report peak-memory /path/to/snapshot.db --metric reserved

# 输出机器可读 JSON
pt-snap report peak-memory /path/to/snapshot.db --json
```

这个 report 命令会组合:

- `memory_peak`
- `allocator_gap`
- `active_memory_callstack_at_event`

并输出人类可读摘要或 JSON。

**示例:**
```bash
Expand Down
5 changes: 4 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,10 @@ where = ["src"]
include = ["pt_snap_cli*"]

[tool.setuptools.package-data]
pt_snap_cli = ["query/templates/*.yaml"]
pt_snap_cli = [
"query/templates/*.yaml",
"query/templates/*/*.yaml",
]

[tool.black]
line-length = 100
Expand Down
137 changes: 136 additions & 1 deletion src/pt_snap_cli/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,9 @@
import json
import shlex
from collections.abc import Mapping
from dataclasses import asdict
from pathlib import Path
from typing import Annotated, NoReturn, cast
from typing import Annotated, Literal, NoReturn, cast

import typer

Expand All @@ -21,8 +22,10 @@
FocusService,
InvalidCategoryError,
InvalidDeviceError,
PeakMemoryReport,
QueryExecutionError,
QueryService,
ReportService,
TemplateNotFoundError,
TemplateRenderError,
)
Expand All @@ -34,6 +37,8 @@
add_completion=True,
context_settings={"help_option_names": ["-h", "--help"]},
)
report_app = typer.Typer(help="Generate memory analysis reports")
app.add_typer(report_app, name="report")


def _focus_service() -> FocusService:
Expand Down Expand Up @@ -356,6 +361,136 @@ def query_database(
raise typer.Exit(1) from None


@report_app.command("peak-memory")
def report_peak_memory(
db_path: Annotated[
Path | None, typer.Argument(help="Path to database file (optional if configured)")
] = None,
device: Annotated[
int | None,
typer.Option(
"--device",
"-d",
help="Device ID to report",
autocompletion=complete_device_ids,
),
] = None,
metric: Annotated[
Literal["active", "allocated", "reserved"],
typer.Option(help="Peak metric to report: active, allocated, or reserved"),
] = "active",
include_static: Annotated[
bool,
typer.Option("--include-static/--exclude-static", help="Include static memory group"),
] = True,
limit: Annotated[int, typer.Option("--limit", "-n", help="Maximum callstack groups")] = 20,
json_output: Annotated[bool, typer.Option("--json", help="Emit machine-readable JSON")] = False,
) -> None:
"""Generate a peak memory attribution report."""
focus_service = _focus_service()
report_service = ReportService(focus_service)
try:
report = report_service.peak_memory_report(
db_path=db_path,
device_id=device,
metric=metric,
include_static=include_static,
limit=limit,
)
except ValueError as e:
_error(str(e))
except FocusFileInvalidError as e:
_error(str(e))
except FocusNotConfiguredError:
typer.secho(
"Error: No database path specified and no database configured.", fg=typer.colors.RED
)
typer.echo(
"Use 'pt-snap focus <database_path>' to set a project database, or provide db_path argument."
)
raise typer.Exit(1) from None
except DatabaseMissingError:
try:
state = focus_service.get_focus(explicit_db_path=db_path, explicit_device_id=device)
except FocusFileInvalidError:
state = None
source = state.source if state is not None else "configured"
path = state.db_path if state is not None else db_path
typer.secho(f"Error: Database from {source} focus not found: {path}", fg=typer.colors.RED)
if state is not None and state.focus_file:
typer.echo(f"Focus file: {state.focus_file}")
typer.echo(
"Use 'pt-snap focus <new_database_path>' to set a new project database, or provide db_path argument."
)
raise typer.Exit(1) from None
except InvalidDeviceError as e:
_error(str(e))
except (
TemplateNotFoundError,
TemplateRenderError,
QueryExecutionError,
DatabaseSchemaError,
) as e:
typer.secho(f"Error generating report: {e}", fg=typer.colors.RED)
raise typer.Exit(1) from None

if json_output:
typer.echo(json.dumps(asdict(report), indent=2))
return

_print_peak_memory_report(report)


def _print_peak_memory_report(report: PeakMemoryReport) -> None:
typer.secho("Peak memory report", fg=typer.colors.GREEN, bold=True)
typer.echo(f"Device: {report.device_id}")
typer.echo(f"Metric: {report.metric}")
typer.echo(f"Event ID: {report.event_id}")
typer.echo()

typer.secho("Peak counters:", fg=typer.colors.GREEN, bold=True)
if report.peak:
typer.echo(
f" allocated: {report.peak.get('peak_allocated')} at event {report.peak.get('peak_allocated_event_id')}"
)
typer.echo(
f" active: {report.peak.get('peak_active')} at event {report.peak.get('peak_active_event_id')}"
)
typer.echo(
f" reserved: {report.peak.get('peak_reserved')} at event {report.peak.get('peak_reserved_event_id')}"
)
else:
typer.echo(" No peak counters found.")
typer.echo()

typer.secho("Allocator gap:", fg=typer.colors.GREEN, bold=True)
if report.allocator_gap:
gap = report.allocator_gap
typer.echo(
f" active peak event: {gap.get('peak_active_event_id')} reserved-active gap={gap.get('reserved_active_gap_at_active_peak')}"
)
typer.echo(
f" allocated peak event: {gap.get('peak_allocated_event_id')} reserved-allocated gap={gap.get('reserved_allocated_gap_at_allocated_peak')}"
)
typer.echo(
f" reserved peak event: {gap.get('peak_reserved_event_id')} reserved-active gap={gap.get('reserved_active_gap_at_reserved_peak')}"
)
typer.echo(f" all peaks same event: {bool(gap.get('all_peaks_same_event'))}")
else:
typer.echo(" No allocator gap data found.")
typer.echo()

typer.secho("Active memory by callstack:", fg=typer.colors.GREEN, bold=True)
if not report.callstack_groups:
typer.echo(" No active memory callstack groups found.")
return
for index, row in enumerate(report.callstack_groups, start=1):
typer.echo(
f" [{index}] {row.get('category')} {row.get('size_bytes')} bytes, {row.get('block_count')} blocks ({row.get('percent_of_active_blocks')}%)"
)
typer.echo(f" {row.get('callstack')}")


@app.command("config")
def show_config(
clear: Annotated[bool, typer.Option("--clear", help="Clear all configuration")] = False,
Expand Down
Loading
Loading