Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions docs/en/skills.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,3 +146,14 @@ same environment still match. Simple `env <name>` shebangs use the current `PATH
Unresolved shebangs, unsupported argument/wrapper semantics (including `env -S`),
or failed identity probes leave `CLI ownership unverified`. A mismatch reports
`CLI belongs to another Python environment`; installation still requires confirmation.

### Range-scoped fragmentation attribution

The fragmentation skill keeps optional active-block attribution at the event
selected by the range-filtered `memory_peak` query. This applies to two-sided
ranges, lower-bound-only ranges, and the default runtime range (`start_id=0`),
which excludes negative synthetic rows. Empty or NULL peaks stop attribution;
blocks allocated before the range but still live at the selected event remain
included. `report peak-memory` selects an unfiltered full-trace peak, so it is
only used for an explicitly requested, separately labelled full-trace comparison
after validating its peak event. Its results do not replace range-scoped evidence.
9 changes: 9 additions & 0 deletions docs/zh/skills.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,3 +136,12 @@ Skill 管理只通过 CLI 提供。
简单的 `env <name>` shebang 按当前 `PATH` 解析;无法解析的 shebang、不支持的参数或
包装器语义(包括 `env -S`),以及身份探测失败,均保留为 `CLI ownership unverified`。
身份不匹配则报告 `CLI belongs to another Python environment`;安装仍需用户确认。

### 范围内碎片归因

碎片诊断 skill 的可选 active 块归因始终使用带范围过滤的 `memory_peak`
查询选出的事件。此规则适用于双界范围、仅下界范围,以及排除负 synthetic
行的默认 runtime 范围(`start_id=0`)。峰值为空或 NULL 时停止归因;在范围
开始前分配、但在所选事件仍存活的块仍纳入归因。`report peak-memory` 会选择
未过滤的全 trace 峰值,因此仅在用户明确要求全 trace 对照、且峰值事件已通过
校验时使用,并单独标注;其结果不能替代范围内证据。
47 changes: 43 additions & 4 deletions skills/pt-snap-memory-fragmentation/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -289,13 +289,46 @@ permitted.

### 5. Optionally describe active blocks at selected events

After verifying the optional surfaces, use either or both of these commands:
#### Range-scoped attribution (including the default runtime range)

Use the `peak_active_event_id` returned by step 1's range-filtered `memory_peak`
as `<peak_active_event_id>` below. This applies to two-sided ranges, lower-bound-only
ranges, and the default `start_id=0` with no upper bound. All of them are filtered
scopes; an omitted upper bound does not make them an unfiltered full trace.

Before attribution, require a non-NULL `peak_active` and an integer, non-negative
`peak_active_event_id` inside the validated range. If the peak query returns no
rows, a NULL peak value/event ID, or an invalid/out-of-range event ID, stop this
phase and report attribution as unavailable. Never fall back to a full-trace
report or substitute another event. For another selected metric or pressure
event, use its event ID from the same range-scoped evidence and validate it too.

```bash
pt-snap query "<db_path>" --device <device_id> --template-use active_memory_callstack_at_event --params '{"event_id":<peak_active_event_id>,"include_static":true,"min_size":0,"top_n":20}' --json
```

The range selects the attribution event, not the allocation dates of live blocks.
Keep blocks allocated before `<range_start>` that are still live at that event;
do not add an allocation-window filter. Record the actual attribution event ID
alongside the range peak and verify they match. Do not use `report peak-memory`
for this branch: it selects its own unfiltered full-trace peak.

#### Separate full-trace comparison (only when explicitly requested)

Use this branch only for an explicitly requested unfiltered full-trace comparison,
labelled separately from the runtime/range diagnosis. First inspect the unfiltered
peak; only run the report if `peak_active` is non-NULL and
`peak_active_event_id` is a valid non-negative integer. Stop this branch on empty,
NULL, or negative synthetic peaks instead of attempting dynamic attribution.

```bash
pt-snap query "<db_path>" --device <device_id> --template-use memory_peak --json
pt-snap report peak-memory "<db_path>" --device <device_id> --metric active --limit 20 --json
pt-snap query "<db_path>" --device <device_id> --template-use active_memory_callstack_at_event --params '{"event_id":<event_id>,"include_static":true,"min_size":0,"top_n":20}'
```

Verify the report's `event_id` matches that unfiltered peak. Never mix this
comparison into range-scoped evidence, even when the peak IDs happen to match.

Use these results only to describe active blocks at a selected event. They do
not attribute reserved bytes, cached bytes, or the `reserved - active` gap to
those callstacks. Keep static and dynamic active-block groups distinct. The
Expand Down Expand Up @@ -377,7 +410,8 @@ experiment needs size-bin or contiguous-free-region evidence.
- Apply one validated event range across every phase so peaks, the curve,
segment operations, optional aggregates, and attribution inputs share a
single scope; reject negative peak event IDs before attribution instead of
attributing them.
attributing them. Range attribution must use the range query's event ID, never
the full-trace report; any explicitly requested full-trace comparison is separate.
- Treat operation counts and size sums as operation volume, not retained bytes.
- Event IDs are ordering markers, not timestamps.
- Separate evidence, inference, unknowns, classifications/confidence, and
Expand All @@ -393,7 +427,12 @@ experiment needs size-bin or contiguous-free-region evidence.
`allocator_gap`.
- The same validated event range was applied consistently to peaks, the
ordered curve, segment operation pairs, optional aggregates, and
attribution inputs.
attribution inputs. The actual attribution event ID matched the selected
range event for two-sided, lower-bound-only, and default `start_id=0` scopes;
empty/NULL peaks stopped attribution without a full-trace fallback. Blocks
allocated before the range but still live at that event were retained.
- Any full-trace report was explicitly requested, checked against its own
unfiltered non-negative peak event, and labelled as a separate comparison.
- The allocated/active/reserved curve used paginated, event-ordered `allocation`
queries, stopping when `has_more` was false.
- Runtime `segment_map`/`segment_unmap` and `segment_alloc`/`segment_free` were
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -289,13 +289,46 @@ permitted.

### 5. Optionally describe active blocks at selected events

After verifying the optional surfaces, use either or both of these commands:
#### Range-scoped attribution (including the default runtime range)

Use the `peak_active_event_id` returned by step 1's range-filtered `memory_peak`
as `<peak_active_event_id>` below. This applies to two-sided ranges, lower-bound-only
ranges, and the default `start_id=0` with no upper bound. All of them are filtered
scopes; an omitted upper bound does not make them an unfiltered full trace.

Before attribution, require a non-NULL `peak_active` and an integer, non-negative
`peak_active_event_id` inside the validated range. If the peak query returns no
rows, a NULL peak value/event ID, or an invalid/out-of-range event ID, stop this
phase and report attribution as unavailable. Never fall back to a full-trace
report or substitute another event. For another selected metric or pressure
event, use its event ID from the same range-scoped evidence and validate it too.

```bash
pt-snap query "<db_path>" --device <device_id> --template-use active_memory_callstack_at_event --params '{"event_id":<peak_active_event_id>,"include_static":true,"min_size":0,"top_n":20}' --json
```

The range selects the attribution event, not the allocation dates of live blocks.
Keep blocks allocated before `<range_start>` that are still live at that event;
do not add an allocation-window filter. Record the actual attribution event ID
alongside the range peak and verify they match. Do not use `report peak-memory`
for this branch: it selects its own unfiltered full-trace peak.

#### Separate full-trace comparison (only when explicitly requested)

Use this branch only for an explicitly requested unfiltered full-trace comparison,
labelled separately from the runtime/range diagnosis. First inspect the unfiltered
peak; only run the report if `peak_active` is non-NULL and
`peak_active_event_id` is a valid non-negative integer. Stop this branch on empty,
NULL, or negative synthetic peaks instead of attempting dynamic attribution.

```bash
pt-snap query "<db_path>" --device <device_id> --template-use memory_peak --json
pt-snap report peak-memory "<db_path>" --device <device_id> --metric active --limit 20 --json
pt-snap query "<db_path>" --device <device_id> --template-use active_memory_callstack_at_event --params '{"event_id":<event_id>,"include_static":true,"min_size":0,"top_n":20}'
```

Verify the report's `event_id` matches that unfiltered peak. Never mix this
comparison into range-scoped evidence, even when the peak IDs happen to match.

Use these results only to describe active blocks at a selected event. They do
not attribute reserved bytes, cached bytes, or the `reserved - active` gap to
those callstacks. Keep static and dynamic active-block groups distinct. The
Expand Down Expand Up @@ -377,7 +410,8 @@ experiment needs size-bin or contiguous-free-region evidence.
- Apply one validated event range across every phase so peaks, the curve,
segment operations, optional aggregates, and attribution inputs share a
single scope; reject negative peak event IDs before attribution instead of
attributing them.
attributing them. Range attribution must use the range query's event ID, never
the full-trace report; any explicitly requested full-trace comparison is separate.
- Treat operation counts and size sums as operation volume, not retained bytes.
- Event IDs are ordering markers, not timestamps.
- Separate evidence, inference, unknowns, classifications/confidence, and
Expand All @@ -393,7 +427,12 @@ experiment needs size-bin or contiguous-free-region evidence.
`allocator_gap`.
- The same validated event range was applied consistently to peaks, the
ordered curve, segment operation pairs, optional aggregates, and
attribution inputs.
attribution inputs. The actual attribution event ID matched the selected
range event for two-sided, lower-bound-only, and default `start_id=0` scopes;
empty/NULL peaks stopped attribution without a full-trace fallback. Blocks
allocated before the range but still live at that event were retained.
- Any full-trace report was explicitly requested, checked against its own
unfiltered non-negative peak event, and labelled as a separate comparison.
- The allocated/active/reserved curve used paginated, event-ordered `allocation`
queries, stopping when `has_more` was false.
- Runtime `segment_map`/`segment_unmap` and `segment_alloc`/`segment_free` were
Expand Down
24 changes: 24 additions & 0 deletions tests/skills/test_memory_fragmentation_contract.py
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,30 @@ def test_memory_fragmentation_skill_preserves_snapshotdb_boundaries() -> None:
assert guardrail in skill


def test_memory_fragmentation_skill_separates_range_and_full_trace_attribution() -> None:
skill = SKILL_PATH.read_text()
attribution = skill.split("### 5.", 1)[1].split("### 6.", 1)[0]
scoped, full_trace = attribution.split("#### Separate full-trace comparison", 1)
normalized = " ".join(scoped.split())

assert "pt-snap report peak-memory" not in scoped
assert '"event_id":<peak_active_event_id>' in scoped
for requirement in (
"step 1's range-filtered `memory_peak`",
"two-sided ranges, lower-bound-only ranges, and the default `start_id=0`",
"integer, non-negative `peak_active_event_id` inside the validated range",
"no rows, a NULL peak value/event ID, or an invalid/out-of-range event ID, stop",
"Never fall back to a full-trace report or substitute another event",
"Keep blocks allocated before `<range_start>` that are still live at that event",
"Record the actual attribution event ID",
):
assert requirement in normalized
assert "only for an explicitly requested unfiltered full-trace comparison" in full_trace
assert "pt-snap report peak-memory" in full_trace
assert "Never mix this" in full_trace
assert "NULL, or negative synthetic peaks" in full_trace


def test_memory_fragmentation_skill_prevents_fragmentation_overclaims() -> None:
skill = SKILL_PATH.read_text()

Expand Down
133 changes: 133 additions & 0 deletions tests/test_fragmentation_query_scope.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
"""Issue #180: execute the skill's peak/attribution commands on distinct scopes."""

import json
import shlex
import sqlite3
from contextlib import closing
from pathlib import Path

import pytest
from typer.testing import CliRunner

from pt_snap_cli.cli import app

SKILL_PATH = Path(__file__).resolve().parents[1] / "skills/pt-snap-memory-fragmentation/SKILL.md"


@pytest.fixture
def scope_db(tmp_path, monkeypatch):
monkeypatch.chdir(tmp_path)
monkeypatch.delenv("PT_SNAP_DB_PATH", raising=False)
monkeypatch.delenv("PT_SNAP_QUERY_TIMEOUT", raising=False)
monkeypatch.setattr(Path, "home", lambda: tmp_path)
path = tmp_path / "scopes.db"
with closing(sqlite3.connect(path)) as conn, conn:
conn.executescript("""
CREATE TABLE dictionary (`table` TEXT, `column` TEXT, `key` TEXT, `value` TEXT);
CREATE TABLE trace_entry_0 (
id INTEGER PRIMARY KEY, allocated INTEGER, active INTEGER,
reserved INTEGER, callstack TEXT
);
CREATE TABLE block_0 (
id INTEGER PRIMARY KEY, size INTEGER, requestedSize INTEGER,
allocEventId INTEGER, freeEventId INTEGER
);
INSERT INTO trace_entry_0 VALUES
(-2, 1000, 1000, 2000, 'synthetic'),
(2, 10, 10, 100, 'before-window'),
(12, 30, 30, 100, 'bounded-peak'),
(20, 10, 10, 100, ''),
(25, 90, 90, 100, 'runtime-peak'),
(30, 10, 10, 100, ''),
(35, 50, 50, 100, 'late-peak'),
(40, 0, 0, 100, ''),
(45, NULL, NULL, NULL, 'unknown-counters');
INSERT INTO block_0 VALUES
(2, 10, 10, 2, 40),
(12, 20, 20, 12, 20),
(25, 80, 80, 25, 30),
(35, 40, 40, 35, 40);
""")
return path


def skill_commands(section):
return [line for line in section.splitlines() if line.startswith("pt-snap ")]


def run_skill_command(command, db_path, **values):
command = command.replace("<db_path>", str(db_path)).replace("<device_id>", "0")
for key, value in values.items():
command = command.replace(f"<{key}>", str(value))
args = shlex.split(command)[1:]
if "--json" not in args:
args.append("--json")
result = CliRunner().invoke(app, args)
assert result.exit_code == 0, result.output
return json.loads(result.stdout)


def range_peak_command(start_id, end_id):
step = SKILL_PATH.read_text().split("### 1. Establish", 1)[1].split("### 2.", 1)[0]
command = skill_commands(step)[0].replace("<range_start>", str(start_id))
if end_id is not None:
# Step 1 explicitly instructs adding this key for two-sided ranges.
command = command.replace("}'", f',"end_id":{end_id}}}' + "'")
return command


@pytest.mark.parametrize(
"start_id,end_id,event_id,active_bytes,callstack",
[
(10, 20, 12, 30, "bounded-peak"),
(30, None, 35, 50, "late-peak"),
(0, None, 25, 90, "runtime-peak"),
],
ids=["two-sided", "lower-bound-only", "default-runtime"],
)
def test_range_peak_drives_actual_attribution(
scope_db, start_id, end_id, event_id, active_bytes, callstack
):
peak = run_skill_command(range_peak_command(start_id, end_id), scope_db)["rows"][0]
assert peak["peak_active_event_id"] == event_id
assert peak["peak_active"] == active_bytes
section = SKILL_PATH.read_text().split("#### Range-scoped attribution", 1)[1]
commands = skill_commands(section.split("#### Separate full-trace comparison", 1)[0])
assert len(commands) == 1
attribution = run_skill_command(commands[0], scope_db, **peak)

# Check the event actually passed through the CLI, not just peak selection.
assert attribution["effective_params"]["event_id"] == event_id
assert event_id >= start_id
assert end_id is None or event_id <= end_id
assert attribution["has_more"] is attribution["truncated"] is False
assert sum(row["size_bytes"] for row in attribution["rows"]) == active_bytes
assert {row["callstack"] for row in attribution["rows"]} == {"before-window", callstack}


@pytest.mark.parametrize("start_id,end_id", [(100, 110), (45, 45)], ids=["empty", "null"])
def test_range_without_usable_peak_cannot_supply_attribution_event(scope_db, start_id, end_id):
peak = run_skill_command(range_peak_command(start_id, end_id), scope_db)["rows"][0]
assert peak["peak_active"] is None
assert peak["peak_active_event_id"] is None
# Stop here per the skill; a report would select a different scope.


def test_explicit_full_trace_comparison_uses_its_own_peak(scope_db):
with closing(sqlite3.connect(scope_db)) as conn, conn:
conn.execute("DELETE FROM trace_entry_0 WHERE id < 0")
section = SKILL_PATH.read_text().split("#### Separate full-trace comparison", 1)[1]
commands = skill_commands(section.split("### 6.", 1)[0])
assert len(commands) == 2
peak = run_skill_command(commands[0], scope_db)["rows"][0]
assert peak["peak_active_event_id"] == 25
report = run_skill_command(commands[1], scope_db)
assert report["event_id"] == peak["peak_active_event_id"]
assert sum(row["size_bytes"] for row in report["callstack_groups"]) == peak["peak_active"]


def test_full_trace_synthetic_peak_requires_stopping_comparison(scope_db):
section = SKILL_PATH.read_text().split("#### Separate full-trace comparison", 1)[1]
peak = run_skill_command(skill_commands(section)[0], scope_db)["rows"][0]
assert peak["peak_active_event_id"] == -2
assert peak["peak_active"] == 1000
Loading