Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 13 additions & 4 deletions skills/pt-snap-memory-leak/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ pt-snap overview '<db_path>' --json

Prerequisite probe budget: previously 7 calls (`metadata` plus six `--template-info` probes). Now 2 calls (`capabilities` + `overview`). Record that reduction when evaluating this skill.

Confirm these templates exist in the capabilities catalog before diagnosis: `memory_peak`, `allocator_gap`, `event`, `block`, `leak_detection`, `active_memory_callstack_at_event`, `preexisting_live`, `freed_block_lifetime`. If the catalog or overview fails, stop and report the exact failure. Do not silently substitute raw SQL for a missing core template.
Confirm these templates exist in the capabilities catalog before diagnosis: `memory_peak`, `allocator_gap`, `event`, `block`, `leak_detection`, `active_memory_callstack_at_event`, `active_blocks_at_event`, `preexisting_live`, `freed_block_lifetime`. If the catalog or overview fails, stop and report the exact failure. Do not silently substitute raw SQL for a missing core template.

## Diagnostic Workflow

Expand Down Expand Up @@ -138,9 +138,18 @@ pt-snap query '<db_path>' --device <device_id> --template-use active_memory_call

Treat this as an occupancy comparison between two independent aggregates, not a block-identity survival test. Each query groups whatever was live at its own event, and `top_n` truncation applies to dynamic callstack groups only; the static and preexisting groups are always returned in full. A peak block may have been freed while an equal-sized later block from the same callstack was live at the final event, which keeps the callstack row stable without any block surviving.

Before claiming that the same blocks persisted across the peak:
- Match representative blocks by identity: the same `id`/address plus allocation event ID must appear live at both events.
- Confirm continuity through lifecycle checks in Step 5; occupancy stability alone is not survival evidence.
Before claiming that the same blocks persisted across the peak, query individual live blocks at both events in the same database and the same device:

```bash
pt-snap query '<db_path>' --device <device_id> --template-use active_blocks_at_event --params '{"event_id":<peak_active_event_id>,"include_static":true,"min_size":0,"order_by":"id","order_dir":"ASC","offset":0}' -n 500 --json
pt-snap query '<db_path>' --device <device_id> --template-use active_blocks_at_event --params '{"event_id":<final_event_id>,"include_static":true,"min_size":0,"order_by":"id","order_dir":"ASC","offset":0}' -n 500 --json
```

- Use a positive page window (`-n 500`), never an unlimited query. These commands return only the first page. For each event independently, advance the `offset` parameter by that page's `returned` count, keeping the event, filters, ordering, database, and device fixed. Continue until `has_more=false` in the JSON response, retaining every preceding page. An offset page still has `truncated=true` because it omits earlier rows; completeness comes from collecting all pages from offset 0 through the terminal page, not from that last page alone. Default `total` is a page count, not the full set; an exact count alone does not retrieve the missing identities.
- Match representative blocks by identity using the exact pair `(id, allocEventId)`, not address, size, or callstack alone. Address reuse does not mean the same allocation survived. Keep `dynamic_live_at_event`, `static`, and `preexisting_live_at_event` categories separate; pre-tracing allocations have unknown allocation history and are not dynamic leak candidates.
- If either event is incompletely paged (or a query fails), label matches as sample-only evidence. Unmatched sample rows are not proven new or released. Do not report a full-set survival rate or claim zero new blocks from samples.
- Only after both complete sets are collected, intersect their `(id, allocEventId)` keys: `matched = peak ∩ final`, `new_at_end = final - peak`, and `released_from_peak = peak - final`. Compute these separately per category. A count-based peak survival rate is `len(matched) / len(peak)`; the share of final blocks already live at peak is `len(matched) / len(final)`. State the denominator and scope, report an empty denominator as unavailable, and do not mix block counts with bytes. Claim zero new blocks only when the complete `new_at_end` set is empty.
- Confirm continuity through lifecycle checks in Step 5; occupancy stability alone is not survival evidence. Check real `freeEventId` values between the two events before describing peak-only blocks as released, and inspect address-event lifecycles for reuse or ambiguous pairing. Even complete identity survival is retention evidence, not proof of a leak.

The template separates blocks without a captured allocation event into their own groups instead of attributing them to callstacks: `[static] allocEventId=-1, freeEventId=-1`, and `[preexisting live] allocEventId=-1` for blocks allocated before tracing that were still live at the analyzed event because their recorded free event came later or does not exist. Pre-tracing blocks whose free event precedes the analyzed event are correctly absent because they were no longer live. Report each group separately instead of absorbing it into a callstack group.

Expand Down
17 changes: 13 additions & 4 deletions src/pt_snap_cli/bundled_skills/pt-snap-memory-leak/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ pt-snap overview '<db_path>' --json

Prerequisite probe budget: previously 7 calls (`metadata` plus six `--template-info` probes). Now 2 calls (`capabilities` + `overview`). Record that reduction when evaluating this skill.

Confirm these templates exist in the capabilities catalog before diagnosis: `memory_peak`, `allocator_gap`, `event`, `block`, `leak_detection`, `active_memory_callstack_at_event`, `preexisting_live`, `freed_block_lifetime`. If the catalog or overview fails, stop and report the exact failure. Do not silently substitute raw SQL for a missing core template.
Confirm these templates exist in the capabilities catalog before diagnosis: `memory_peak`, `allocator_gap`, `event`, `block`, `leak_detection`, `active_memory_callstack_at_event`, `active_blocks_at_event`, `preexisting_live`, `freed_block_lifetime`. If the catalog or overview fails, stop and report the exact failure. Do not silently substitute raw SQL for a missing core template.

## Diagnostic Workflow

Expand Down Expand Up @@ -138,9 +138,18 @@ pt-snap query '<db_path>' --device <device_id> --template-use active_memory_call

Treat this as an occupancy comparison between two independent aggregates, not a block-identity survival test. Each query groups whatever was live at its own event, and `top_n` truncation applies to dynamic callstack groups only; the static and preexisting groups are always returned in full. A peak block may have been freed while an equal-sized later block from the same callstack was live at the final event, which keeps the callstack row stable without any block surviving.

Before claiming that the same blocks persisted across the peak:
- Match representative blocks by identity: the same `id`/address plus allocation event ID must appear live at both events.
- Confirm continuity through lifecycle checks in Step 5; occupancy stability alone is not survival evidence.
Before claiming that the same blocks persisted across the peak, query individual live blocks at both events in the same database and the same device:

```bash
pt-snap query '<db_path>' --device <device_id> --template-use active_blocks_at_event --params '{"event_id":<peak_active_event_id>,"include_static":true,"min_size":0,"order_by":"id","order_dir":"ASC","offset":0}' -n 500 --json
pt-snap query '<db_path>' --device <device_id> --template-use active_blocks_at_event --params '{"event_id":<final_event_id>,"include_static":true,"min_size":0,"order_by":"id","order_dir":"ASC","offset":0}' -n 500 --json
```

- Use a positive page window (`-n 500`), never an unlimited query. These commands return only the first page. For each event independently, advance the `offset` parameter by that page's `returned` count, keeping the event, filters, ordering, database, and device fixed. Continue until `has_more=false` in the JSON response, retaining every preceding page. An offset page still has `truncated=true` because it omits earlier rows; completeness comes from collecting all pages from offset 0 through the terminal page, not from that last page alone. Default `total` is a page count, not the full set; an exact count alone does not retrieve the missing identities.
- Match representative blocks by identity using the exact pair `(id, allocEventId)`, not address, size, or callstack alone. Address reuse does not mean the same allocation survived. Keep `dynamic_live_at_event`, `static`, and `preexisting_live_at_event` categories separate; pre-tracing allocations have unknown allocation history and are not dynamic leak candidates.
- If either event is incompletely paged (or a query fails), label matches as sample-only evidence. Unmatched sample rows are not proven new or released. Do not report a full-set survival rate or claim zero new blocks from samples.
- Only after both complete sets are collected, intersect their `(id, allocEventId)` keys: `matched = peak ∩ final`, `new_at_end = final - peak`, and `released_from_peak = peak - final`. Compute these separately per category. A count-based peak survival rate is `len(matched) / len(peak)`; the share of final blocks already live at peak is `len(matched) / len(final)`. State the denominator and scope, report an empty denominator as unavailable, and do not mix block counts with bytes. Claim zero new blocks only when the complete `new_at_end` set is empty.
- Confirm continuity through lifecycle checks in Step 5; occupancy stability alone is not survival evidence. Check real `freeEventId` values between the two events before describing peak-only blocks as released, and inspect address-event lifecycles for reuse or ambiguous pairing. Even complete identity survival is retention evidence, not proof of a leak.

The template separates blocks without a captured allocation event into their own groups instead of attributing them to callstacks: `[static] allocEventId=-1, freeEventId=-1`, and `[preexisting live] allocEventId=-1` for blocks allocated before tracing that were still live at the analyzed event because their recorded free event came later or does not exist. Pre-tracing blocks whose free event precedes the analyzed event are correctly absent because they were no longer live. Report each group separately instead of absorbing it into a callstack group.

Expand Down
154 changes: 154 additions & 0 deletions tests/query/test_peak_memory_templates.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
"""Tests for peak memory attribution query templates."""

import json
import sqlite3
from pathlib import Path

import pytest
from typer.testing import CliRunner

from pt_snap_cli.cli import app
from pt_snap_cli.context import Context
from pt_snap_cli.query.executor import QueryExecutor
from pt_snap_cli.query.registry import (
Expand Down Expand Up @@ -408,3 +411,154 @@ def test_callstack_percent_metadata_describes_truncated_denominator() -> None:
assert percent["units"] == "percent"
assert "top_n truncation of dynamic" in percent["denominator"]
assert percent["scope"] == "mixed"


@pytest.mark.parametrize("block_count", [1, 501])
@pytest.mark.parametrize("replace_last", [False, True], ids=["survival", "address-reuse"])
def test_peak_final_identity_matching_requires_complete_pages(
peak_memory_db: Path,
monkeypatch: pytest.MonkeyPatch,
block_count: int,
replace_last: bool,
) -> None:
"""Equal callstack occupancy can hide a replacement beyond the first 500 rows."""
monkeypatch.chdir(peak_memory_db.parent)
monkeypatch.setattr(Path, "home", lambda: peak_memory_db.parent)
monkeypatch.delenv("PT_SNAP_DB_PATH", raising=False)
monkeypatch.delenv("PT_SNAP_QUERY_TIMEOUT", raising=False)
peak_event = block_count
final_event = block_count + 4
conn = sqlite3.connect(peak_memory_db)
# Reuse the existing v2 fixture schema and train.py:10 callstack.
conn.execute("DELETE FROM block_0")
conn.execute("DELETE FROM trace_entry_0")
conn.executemany(
"INSERT INTO trace_entry_0 VALUES (?, 4, ?, 1024, 0, ?, ?, 1048576, 0)",
[(i, i * 4096, i * 1024, i * 1024) for i in range(1, block_count + 1)],
)
conn.executemany(
"INSERT INTO block_0 VALUES (?, ?, 1024, 1000, 1, ?, -1)",
[(i, i * 4096, i) for i in range(1, block_count + 1)],
)
if replace_last:
conn.execute(
"UPDATE block_0 SET freeEventId = ? WHERE id = ?",
(peak_event + 2, block_count),
)
conn.executemany(
"INSERT INTO trace_entry_0 VALUES (?, ?, ?, 1024, 0, ?, ?, 1048576, 0)",
[
(
peak_event + 1,
5,
block_count * 4096,
(block_count - 1) * 1024,
block_count * 1024,
),
(
peak_event + 2,
6,
block_count * 4096,
(block_count - 1) * 1024,
(block_count - 1) * 1024,
),
(peak_event + 3, 4, block_count * 4096, block_count * 1024, block_count * 1024),
],
)
conn.execute(
"INSERT INTO block_0 VALUES (?, ?, 1024, 1000, 1, ?, -1)",
(peak_event + 3, block_count * 4096, peak_event + 3),
)
conn.execute(
"INSERT INTO trace_entry_0 VALUES (?, 7, 0, 0, 0, ?, ?, 1048576, NULL)",
(final_event, block_count * 1024, block_count * 1024),
)
conn.commit()
conn.close()

aggregates = [
_execute_template(
peak_memory_db,
"active_memory_callstack_at_event",
{"event_id": event, "include_static": True, "min_size": 0, "top_n": 20},
)
for event in (peak_event, final_event)
]
assert aggregates[0] == aggregates[1]
assert aggregates[0][0]["block_count"] == block_count
assert aggregates[0][0]["size_bytes"] == block_count * 1024

runner = CliRunner()
sets = []
first_pages = []
for event in (peak_event, final_event):
collected = []
offsets = []
while True:
offset = len(collected)
offsets.append(offset)
assert len(offsets) <= 2, "pagination did not terminate"
result = runner.invoke(
app,
[
"query",
str(peak_memory_db),
"--device",
"0",
"--template-use",
"active_blocks_at_event",
"--params",
json.dumps(
{
"event_id": event,
"include_static": True,
"min_size": 0,
"order_by": "id",
"order_dir": "ASC",
"offset": offset,
}
),
"-n",
"500",
"--json",
],
)
assert result.exit_code == 0, result.stdout
page = json.loads(result.stdout)
assert page["db_path"] == str(peak_memory_db)
assert page["device_id"] == 0
assert page["returned"] == len(page["rows"]) <= 500
assert page["total"] == page["returned"]
if offset == 0:
first_pages.append({(row["id"], row["allocEventId"]) for row in page["rows"]})
assert page["has_more"] is (block_count > 500)
collected.extend(page["rows"])
if not page["has_more"]:
assert page["truncated"] is (offset > 0)
break
assert page["truncated"] is True
assert offsets == ([0, 500] if block_count > 500 else [0])
assert len(collected) == block_count
identities = {(row["id"], row["allocEventId"]): row for row in collected}
assert len(identities) == block_count # No duplicated or skipped identities.
assert {row["category"] for row in collected} == {"dynamic_live_at_event"}
sets.append(identities)

peak, final = sets
matched = peak.keys() & final.keys()
new_at_end = final.keys() - peak.keys()
released = peak.keys() - final.keys()
if block_count > 500:
assert first_pages[0] == first_pages[1] # A sample hides the replacement.
if replace_last:
assert len(matched) == block_count - 1
assert new_at_end == {(peak_event + 3, peak_event + 3)}
assert released == {(block_count, block_count)}
old = peak[next(iter(released))]
new = final[next(iter(new_at_end))]
assert old["address"] == new["address"]
assert peak_event < old["freeEventId"] < new["allocEventId"] <= final_event
else:
assert len(matched) == block_count
assert new_at_end == released == set()
assert not (peak_memory_db.parent / ".pt-snap").exists()
54 changes: 54 additions & 0 deletions tests/skills/test_memory_leak_contract.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import json
import shlex
from pathlib import Path

SKILL_PATH = Path("skills/pt-snap-memory-leak/SKILL.md")
Expand All @@ -17,6 +19,7 @@ def test_memory_leak_skill_uses_current_pt_snap_surfaces() -> None:
"block",
"leak_detection",
"active_memory_callstack_at_event",
"active_blocks_at_event",
"preexisting_live",
"freed_block_lifetime",
):
Expand Down Expand Up @@ -95,6 +98,57 @@ def test_memory_leak_skill_matches_current_preexisting_group_semantics() -> None
assert "counts only blocks with `allocEventId != -1`" not in skill


def test_memory_leak_step4_queries_both_identity_sets_with_bounded_json_pages() -> None:
skill = SKILL_PATH.read_text()
prerequisite, workflow = skill.split("## Diagnostic Workflow", 1)
assert "`active_blocks_at_event`" in prerequisite
step4 = workflow.split("### 4.", 1)[1].split("### 5.", 1)[0]
commands = [
shlex.split(line)
for line in step4.splitlines()
if line.startswith("pt-snap query") and "--template-use active_blocks_at_event" in line
]
assert len(commands) == 2
for command, event in zip(
commands, ("<peak_active_event_id>", "<final_event_id>"), strict=True
):
assert command[:5] == ["pt-snap", "query", "<db_path>", "--device", "<device_id>"]
assert "--json" in command
assert int(command[command.index("-n") + 1]) == 500
params = json.loads(command[command.index("--params") + 1].replace(event, "42"))
assert params == {
"event_id": 42,
"include_static": True,
"min_size": 0,
"order_by": "id",
"order_dir": "ASC",
"offset": 0,
}
for requirement in (
"same database and the same device",
"advance the `offset` parameter by that page's `returned` count",
"Continue until `has_more=false`",
"An offset page still has `truncated=true`",
"collecting all pages from offset 0 through the terminal page",
"exact pair `(id, allocEventId)`",
"sample-only evidence",
"Unmatched sample rows are not proven new or released",
"Do not report a full-set survival rate or claim zero new blocks from samples",
"Only after both complete sets are collected",
"`matched = peak ∩ final`",
"`new_at_end = final - peak`",
"`released_from_peak = peak - final`",
"Compute these separately per category",
"`len(matched) / len(peak)`",
"`len(matched) / len(final)`",
"empty denominator as unavailable",
"lifecycle checks in Step 5",
"Check real `freeEventId` values",
"not proof of a leak",
):
assert requirement in step4


def test_memory_leak_skill_cross_checks_preexisting_bucket_exactly() -> None:
skill = SKILL_PATH.read_text()

Expand Down
Loading