diff --git a/skills/pt-snap-memory-leak/SKILL.md b/skills/pt-snap-memory-leak/SKILL.md index 9af2fd9..a918354 100644 --- a/skills/pt-snap-memory-leak/SKILL.md +++ b/skills/pt-snap-memory-leak/SKILL.md @@ -80,7 +80,7 @@ pt-snap overview '' --json Prerequisite probe budget: previously 7 calls (`metadata` plus six `--template-info` probes). Now 2 calls (`capabilities` + `overview`). Record that reduction when evaluating this skill. -Confirm these templates exist in the capabilities catalog before diagnosis: `memory_peak`, `allocator_gap`, `event`, `block`, `leak_detection`, `active_memory_callstack_at_event`, `preexisting_live`, `freed_block_lifetime`. If the catalog or overview fails, stop and report the exact failure. Do not silently substitute raw SQL for a missing core template. +Confirm these templates exist in the capabilities catalog before diagnosis: `memory_peak`, `allocator_gap`, `event`, `block`, `leak_detection`, `active_memory_callstack_at_event`, `active_blocks_at_event`, `preexisting_live`, `freed_block_lifetime`. If the catalog or overview fails, stop and report the exact failure. Do not silently substitute raw SQL for a missing core template. ## Diagnostic Workflow @@ -138,9 +138,18 @@ pt-snap query '' --device --template-use active_memory_call Treat this as an occupancy comparison between two independent aggregates, not a block-identity survival test. Each query groups whatever was live at its own event, and `top_n` truncation applies to dynamic callstack groups only; the static and preexisting groups are always returned in full. A peak block may have been freed while an equal-sized later block from the same callstack was live at the final event, which keeps the callstack row stable without any block surviving. -Before claiming that the same blocks persisted across the peak: -- Match representative blocks by identity: the same `id`/address plus allocation event ID must appear live at both events. -- Confirm continuity through lifecycle checks in Step 5; occupancy stability alone is not survival evidence. +Before claiming that the same blocks persisted across the peak, query individual live blocks at both events in the same database and the same device: + +```bash +pt-snap query '' --device --template-use active_blocks_at_event --params '{"event_id":,"include_static":true,"min_size":0,"order_by":"id","order_dir":"ASC","offset":0}' -n 500 --json +pt-snap query '' --device --template-use active_blocks_at_event --params '{"event_id":,"include_static":true,"min_size":0,"order_by":"id","order_dir":"ASC","offset":0}' -n 500 --json +``` + +- Use a positive page window (`-n 500`), never an unlimited query. These commands return only the first page. For each event independently, advance the `offset` parameter by that page's `returned` count, keeping the event, filters, ordering, database, and device fixed. Continue until `has_more=false` in the JSON response, retaining every preceding page. An offset page still has `truncated=true` because it omits earlier rows; completeness comes from collecting all pages from offset 0 through the terminal page, not from that last page alone. Default `total` is a page count, not the full set; an exact count alone does not retrieve the missing identities. +- Match representative blocks by identity using the exact pair `(id, allocEventId)`, not address, size, or callstack alone. Address reuse does not mean the same allocation survived. Keep `dynamic_live_at_event`, `static`, and `preexisting_live_at_event` categories separate; pre-tracing allocations have unknown allocation history and are not dynamic leak candidates. +- If either event is incompletely paged (or a query fails), label matches as sample-only evidence. Unmatched sample rows are not proven new or released. Do not report a full-set survival rate or claim zero new blocks from samples. +- Only after both complete sets are collected, intersect their `(id, allocEventId)` keys: `matched = peak ∩ final`, `new_at_end = final - peak`, and `released_from_peak = peak - final`. Compute these separately per category. A count-based peak survival rate is `len(matched) / len(peak)`; the share of final blocks already live at peak is `len(matched) / len(final)`. State the denominator and scope, report an empty denominator as unavailable, and do not mix block counts with bytes. Claim zero new blocks only when the complete `new_at_end` set is empty. +- Confirm continuity through lifecycle checks in Step 5; occupancy stability alone is not survival evidence. Check real `freeEventId` values between the two events before describing peak-only blocks as released, and inspect address-event lifecycles for reuse or ambiguous pairing. Even complete identity survival is retention evidence, not proof of a leak. The template separates blocks without a captured allocation event into their own groups instead of attributing them to callstacks: `[static] allocEventId=-1, freeEventId=-1`, and `[preexisting live] allocEventId=-1` for blocks allocated before tracing that were still live at the analyzed event because their recorded free event came later or does not exist. Pre-tracing blocks whose free event precedes the analyzed event are correctly absent because they were no longer live. Report each group separately instead of absorbing it into a callstack group. diff --git a/src/pt_snap_cli/bundled_skills/pt-snap-memory-leak/SKILL.md b/src/pt_snap_cli/bundled_skills/pt-snap-memory-leak/SKILL.md index 9af2fd9..a918354 100644 --- a/src/pt_snap_cli/bundled_skills/pt-snap-memory-leak/SKILL.md +++ b/src/pt_snap_cli/bundled_skills/pt-snap-memory-leak/SKILL.md @@ -80,7 +80,7 @@ pt-snap overview '' --json Prerequisite probe budget: previously 7 calls (`metadata` plus six `--template-info` probes). Now 2 calls (`capabilities` + `overview`). Record that reduction when evaluating this skill. -Confirm these templates exist in the capabilities catalog before diagnosis: `memory_peak`, `allocator_gap`, `event`, `block`, `leak_detection`, `active_memory_callstack_at_event`, `preexisting_live`, `freed_block_lifetime`. If the catalog or overview fails, stop and report the exact failure. Do not silently substitute raw SQL for a missing core template. +Confirm these templates exist in the capabilities catalog before diagnosis: `memory_peak`, `allocator_gap`, `event`, `block`, `leak_detection`, `active_memory_callstack_at_event`, `active_blocks_at_event`, `preexisting_live`, `freed_block_lifetime`. If the catalog or overview fails, stop and report the exact failure. Do not silently substitute raw SQL for a missing core template. ## Diagnostic Workflow @@ -138,9 +138,18 @@ pt-snap query '' --device --template-use active_memory_call Treat this as an occupancy comparison between two independent aggregates, not a block-identity survival test. Each query groups whatever was live at its own event, and `top_n` truncation applies to dynamic callstack groups only; the static and preexisting groups are always returned in full. A peak block may have been freed while an equal-sized later block from the same callstack was live at the final event, which keeps the callstack row stable without any block surviving. -Before claiming that the same blocks persisted across the peak: -- Match representative blocks by identity: the same `id`/address plus allocation event ID must appear live at both events. -- Confirm continuity through lifecycle checks in Step 5; occupancy stability alone is not survival evidence. +Before claiming that the same blocks persisted across the peak, query individual live blocks at both events in the same database and the same device: + +```bash +pt-snap query '' --device --template-use active_blocks_at_event --params '{"event_id":,"include_static":true,"min_size":0,"order_by":"id","order_dir":"ASC","offset":0}' -n 500 --json +pt-snap query '' --device --template-use active_blocks_at_event --params '{"event_id":,"include_static":true,"min_size":0,"order_by":"id","order_dir":"ASC","offset":0}' -n 500 --json +``` + +- Use a positive page window (`-n 500`), never an unlimited query. These commands return only the first page. For each event independently, advance the `offset` parameter by that page's `returned` count, keeping the event, filters, ordering, database, and device fixed. Continue until `has_more=false` in the JSON response, retaining every preceding page. An offset page still has `truncated=true` because it omits earlier rows; completeness comes from collecting all pages from offset 0 through the terminal page, not from that last page alone. Default `total` is a page count, not the full set; an exact count alone does not retrieve the missing identities. +- Match representative blocks by identity using the exact pair `(id, allocEventId)`, not address, size, or callstack alone. Address reuse does not mean the same allocation survived. Keep `dynamic_live_at_event`, `static`, and `preexisting_live_at_event` categories separate; pre-tracing allocations have unknown allocation history and are not dynamic leak candidates. +- If either event is incompletely paged (or a query fails), label matches as sample-only evidence. Unmatched sample rows are not proven new or released. Do not report a full-set survival rate or claim zero new blocks from samples. +- Only after both complete sets are collected, intersect their `(id, allocEventId)` keys: `matched = peak ∩ final`, `new_at_end = final - peak`, and `released_from_peak = peak - final`. Compute these separately per category. A count-based peak survival rate is `len(matched) / len(peak)`; the share of final blocks already live at peak is `len(matched) / len(final)`. State the denominator and scope, report an empty denominator as unavailable, and do not mix block counts with bytes. Claim zero new blocks only when the complete `new_at_end` set is empty. +- Confirm continuity through lifecycle checks in Step 5; occupancy stability alone is not survival evidence. Check real `freeEventId` values between the two events before describing peak-only blocks as released, and inspect address-event lifecycles for reuse or ambiguous pairing. Even complete identity survival is retention evidence, not proof of a leak. The template separates blocks without a captured allocation event into their own groups instead of attributing them to callstacks: `[static] allocEventId=-1, freeEventId=-1`, and `[preexisting live] allocEventId=-1` for blocks allocated before tracing that were still live at the analyzed event because their recorded free event came later or does not exist. Pre-tracing blocks whose free event precedes the analyzed event are correctly absent because they were no longer live. Report each group separately instead of absorbing it into a callstack group. diff --git a/tests/query/test_peak_memory_templates.py b/tests/query/test_peak_memory_templates.py index 670d28b..dc46626 100644 --- a/tests/query/test_peak_memory_templates.py +++ b/tests/query/test_peak_memory_templates.py @@ -1,10 +1,13 @@ """Tests for peak memory attribution query templates.""" +import json import sqlite3 from pathlib import Path import pytest +from typer.testing import CliRunner +from pt_snap_cli.cli import app from pt_snap_cli.context import Context from pt_snap_cli.query.executor import QueryExecutor from pt_snap_cli.query.registry import ( @@ -408,3 +411,154 @@ def test_callstack_percent_metadata_describes_truncated_denominator() -> None: assert percent["units"] == "percent" assert "top_n truncation of dynamic" in percent["denominator"] assert percent["scope"] == "mixed" + + +@pytest.mark.parametrize("block_count", [1, 501]) +@pytest.mark.parametrize("replace_last", [False, True], ids=["survival", "address-reuse"]) +def test_peak_final_identity_matching_requires_complete_pages( + peak_memory_db: Path, + monkeypatch: pytest.MonkeyPatch, + block_count: int, + replace_last: bool, +) -> None: + """Equal callstack occupancy can hide a replacement beyond the first 500 rows.""" + monkeypatch.chdir(peak_memory_db.parent) + monkeypatch.setattr(Path, "home", lambda: peak_memory_db.parent) + monkeypatch.delenv("PT_SNAP_DB_PATH", raising=False) + monkeypatch.delenv("PT_SNAP_QUERY_TIMEOUT", raising=False) + peak_event = block_count + final_event = block_count + 4 + conn = sqlite3.connect(peak_memory_db) + # Reuse the existing v2 fixture schema and train.py:10 callstack. + conn.execute("DELETE FROM block_0") + conn.execute("DELETE FROM trace_entry_0") + conn.executemany( + "INSERT INTO trace_entry_0 VALUES (?, 4, ?, 1024, 0, ?, ?, 1048576, 0)", + [(i, i * 4096, i * 1024, i * 1024) for i in range(1, block_count + 1)], + ) + conn.executemany( + "INSERT INTO block_0 VALUES (?, ?, 1024, 1000, 1, ?, -1)", + [(i, i * 4096, i) for i in range(1, block_count + 1)], + ) + if replace_last: + conn.execute( + "UPDATE block_0 SET freeEventId = ? WHERE id = ?", + (peak_event + 2, block_count), + ) + conn.executemany( + "INSERT INTO trace_entry_0 VALUES (?, ?, ?, 1024, 0, ?, ?, 1048576, 0)", + [ + ( + peak_event + 1, + 5, + block_count * 4096, + (block_count - 1) * 1024, + block_count * 1024, + ), + ( + peak_event + 2, + 6, + block_count * 4096, + (block_count - 1) * 1024, + (block_count - 1) * 1024, + ), + (peak_event + 3, 4, block_count * 4096, block_count * 1024, block_count * 1024), + ], + ) + conn.execute( + "INSERT INTO block_0 VALUES (?, ?, 1024, 1000, 1, ?, -1)", + (peak_event + 3, block_count * 4096, peak_event + 3), + ) + conn.execute( + "INSERT INTO trace_entry_0 VALUES (?, 7, 0, 0, 0, ?, ?, 1048576, NULL)", + (final_event, block_count * 1024, block_count * 1024), + ) + conn.commit() + conn.close() + + aggregates = [ + _execute_template( + peak_memory_db, + "active_memory_callstack_at_event", + {"event_id": event, "include_static": True, "min_size": 0, "top_n": 20}, + ) + for event in (peak_event, final_event) + ] + assert aggregates[0] == aggregates[1] + assert aggregates[0][0]["block_count"] == block_count + assert aggregates[0][0]["size_bytes"] == block_count * 1024 + + runner = CliRunner() + sets = [] + first_pages = [] + for event in (peak_event, final_event): + collected = [] + offsets = [] + while True: + offset = len(collected) + offsets.append(offset) + assert len(offsets) <= 2, "pagination did not terminate" + result = runner.invoke( + app, + [ + "query", + str(peak_memory_db), + "--device", + "0", + "--template-use", + "active_blocks_at_event", + "--params", + json.dumps( + { + "event_id": event, + "include_static": True, + "min_size": 0, + "order_by": "id", + "order_dir": "ASC", + "offset": offset, + } + ), + "-n", + "500", + "--json", + ], + ) + assert result.exit_code == 0, result.stdout + page = json.loads(result.stdout) + assert page["db_path"] == str(peak_memory_db) + assert page["device_id"] == 0 + assert page["returned"] == len(page["rows"]) <= 500 + assert page["total"] == page["returned"] + if offset == 0: + first_pages.append({(row["id"], row["allocEventId"]) for row in page["rows"]}) + assert page["has_more"] is (block_count > 500) + collected.extend(page["rows"]) + if not page["has_more"]: + assert page["truncated"] is (offset > 0) + break + assert page["truncated"] is True + assert offsets == ([0, 500] if block_count > 500 else [0]) + assert len(collected) == block_count + identities = {(row["id"], row["allocEventId"]): row for row in collected} + assert len(identities) == block_count # No duplicated or skipped identities. + assert {row["category"] for row in collected} == {"dynamic_live_at_event"} + sets.append(identities) + + peak, final = sets + matched = peak.keys() & final.keys() + new_at_end = final.keys() - peak.keys() + released = peak.keys() - final.keys() + if block_count > 500: + assert first_pages[0] == first_pages[1] # A sample hides the replacement. + if replace_last: + assert len(matched) == block_count - 1 + assert new_at_end == {(peak_event + 3, peak_event + 3)} + assert released == {(block_count, block_count)} + old = peak[next(iter(released))] + new = final[next(iter(new_at_end))] + assert old["address"] == new["address"] + assert peak_event < old["freeEventId"] < new["allocEventId"] <= final_event + else: + assert len(matched) == block_count + assert new_at_end == released == set() + assert not (peak_memory_db.parent / ".pt-snap").exists() diff --git a/tests/skills/test_memory_leak_contract.py b/tests/skills/test_memory_leak_contract.py index ae0dbf1..415a663 100644 --- a/tests/skills/test_memory_leak_contract.py +++ b/tests/skills/test_memory_leak_contract.py @@ -1,3 +1,5 @@ +import json +import shlex from pathlib import Path SKILL_PATH = Path("skills/pt-snap-memory-leak/SKILL.md") @@ -17,6 +19,7 @@ def test_memory_leak_skill_uses_current_pt_snap_surfaces() -> None: "block", "leak_detection", "active_memory_callstack_at_event", + "active_blocks_at_event", "preexisting_live", "freed_block_lifetime", ): @@ -95,6 +98,57 @@ def test_memory_leak_skill_matches_current_preexisting_group_semantics() -> None assert "counts only blocks with `allocEventId != -1`" not in skill +def test_memory_leak_step4_queries_both_identity_sets_with_bounded_json_pages() -> None: + skill = SKILL_PATH.read_text() + prerequisite, workflow = skill.split("## Diagnostic Workflow", 1) + assert "`active_blocks_at_event`" in prerequisite + step4 = workflow.split("### 4.", 1)[1].split("### 5.", 1)[0] + commands = [ + shlex.split(line) + for line in step4.splitlines() + if line.startswith("pt-snap query") and "--template-use active_blocks_at_event" in line + ] + assert len(commands) == 2 + for command, event in zip( + commands, ("", ""), strict=True + ): + assert command[:5] == ["pt-snap", "query", "", "--device", ""] + assert "--json" in command + assert int(command[command.index("-n") + 1]) == 500 + params = json.loads(command[command.index("--params") + 1].replace(event, "42")) + assert params == { + "event_id": 42, + "include_static": True, + "min_size": 0, + "order_by": "id", + "order_dir": "ASC", + "offset": 0, + } + for requirement in ( + "same database and the same device", + "advance the `offset` parameter by that page's `returned` count", + "Continue until `has_more=false`", + "An offset page still has `truncated=true`", + "collecting all pages from offset 0 through the terminal page", + "exact pair `(id, allocEventId)`", + "sample-only evidence", + "Unmatched sample rows are not proven new or released", + "Do not report a full-set survival rate or claim zero new blocks from samples", + "Only after both complete sets are collected", + "`matched = peak ∩ final`", + "`new_at_end = final - peak`", + "`released_from_peak = peak - final`", + "Compute these separately per category", + "`len(matched) / len(peak)`", + "`len(matched) / len(final)`", + "empty denominator as unavailable", + "lifecycle checks in Step 5", + "Check real `freeEventId` values", + "not proof of a leak", + ): + assert requirement in step4 + + def test_memory_leak_skill_cross_checks_preexisting_bucket_exactly() -> None: skill = SKILL_PATH.read_text()