Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -1,2 +1,9 @@
tests/fixtures/snapshots/snapshot_import_131k_sanitized.pickle filter=lfs diff=lfs merge=lfs -text
tests/fixtures/snapshots/snapshot_import_628k_sanitized.pickle filter=lfs diff=lfs merge=lfs -text

# Exact-hash interop fixture bytes must survive core.autocrlf checkouts.
tests/fixtures/interop/generate_three_shard.py text eol=lf
tests/fixtures/interop/three_shard_free_requested.json text eol=lf
tests/fixtures/interop/checksums.json text eol=lf
tests/fixtures/interop/three_shard_free_requested.pkl.msinsight/manifest.json text eol=lf
tests/fixtures/interop/three_shard_free_requested.pkl.msinsight/device_0/*.db -text
48 changes: 48 additions & 0 deletions docs/en/interop-acceptance.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,54 @@ Reports use gap/attribution/active coverage at the SAME selected metric event.
One query/report shares diminishing time and cumulative 100000 fetched/output rows
/64 MiB serialized values, NOT RSS. Failure yields no partial false global result.

## Three-shard free-requested fixture

[pt-snap-cli #249](https://github.com/codeasier/pt-snap-cli/issues/249) adds a
minimal, synthetic producer fixture under
[`tests/fixtures/interop/`](../../tests/fixtures/interop/README.md): one device,
three real events and three shards, generated by the real CLI with
`--events-per-slice 1`. Allocation 0, free_requested 1 and free_completed 2 each
belong to exactly one shard. There is no new committed executable pickle or
third-party producer code. Source, generation identity, privacy/rights review,
size/hash records and limitations are in the linked provenance record.

```bash
PYTHONPATH="$PWD/src" python -m pytest tests/test_fixture_provenance.py
PYTHONPATH="$PWD/src" python -m pytest tests/test_interop_three_shard.py
PYTHONPATH="$PWD/src" python tests/fixtures/interop/generate_three_shard.py \
--output /tmp/NEW-three-shard-interop
```

Use an absent output directory. The generator creates its own synthetic pickle
from the inspectable JSON and invokes `pt-snap import` for both `msinsight`
(compatibility-v1, inline callstack) and `pt-snap-native-v2` (interned callstack),
with `--device 0 --events-per-slice 1 --no-focus`. It does not open the GUI or run
an original producer. The committed v1 artifact is checksum-verified before
read-only tests; fresh output reproduces its physical semantics, not necessarily
its timestamp/path/version-dependent bytes.

The regression checks event ownership, all nine physical event fields, complete
stack text and block references. The free-requested row has ID 1, action 5,
address 4096, size 16, stream 7 and allocated/active/reserved counters 0/16/64
bytes (the consumer's Caching Allocated/Active/Reserved fields). Block 0 retains
`allocEventId=0`, `freeEventId=2`; `freeEventId` identifies **completed** free.
The existing current-shard-only consumer predicate misses event 1 in shard 0/2.
An independent SQL oracle searches manifest candidates intersecting
`allocEventId < id < freeEventId`, or through the device's final real event when
`freeEventId < 0`, then filters by action/address and the same bounds. Separate
v1/v2 cases prove address reuse cannot attach a later lifetime's request and
open-ended pending-free/allocated lifetimes yield the right event or no event.
The older OOM-containing nine-event `lifecycle_data` is intentionally not used as
this minimal v1 fixture; existing full-replay coverage is retained.

For a separately authorized GUI retest, use the newly generated original
`three_shard_free_requested.pkl` beside its generated `.msinsight` directory.
Do not expose the immutable committed fixture to a consumer that writes derived
cache tables. Native-v2 is not a claimed msinsight GUI input. This work proves
producer/fixture contracts only: it neither fixes nor verifies
[msinsight #625](https://gitcode.com/Ascend/msinsight/issues/625), and does not
supersede the consumer limitation in the dated acceptance record below.

## Performance: same reviewed fixture, three modes

The bounded measured fixture is `snapshot_expandable.pkl`, 581140 bytes,
Expand Down
40 changes: 40 additions & 0 deletions docs/zh/interop-acceptance.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,46 @@ missing label、非NULL空文本活动组不能混同。`callstack_analysis` 语
共用递减时间及累计100000 fetched/output行、64 MiB序列化值,**不是 RSS**;失败不能给出
部分的虚假全局结论。

## 三片 free-requested fixture

[pt-snap-cli #249](https://github.com/codeasier/pt-snap-cli/issues/249) 在
[`tests/fixtures/interop/`](../../tests/fixtures/interop/README.md) 增加最小合成生产端
fixture:单设备、三个真实事件、三个分片,由真实 CLI 配合 `--events-per-slice 1`
生成。allocation 0、free_requested 1、free_completed 2 各只属于一个分片。
不新增已提交的可执行 pickle,也不引入第三方生产端代码。来源、生成身份、隐私/权利审核、
大小/hash 和限制均记录在该目录的 provenance 中。

```bash
PYTHONPATH="$PWD/src" python -m pytest tests/test_fixture_provenance.py
PYTHONPATH="$PWD/src" python -m pytest tests/test_interop_three_shard.py
PYTHONPATH="$PWD/src" python tests/fixtures/interop/generate_three_shard.py \
--output /tmp/NEW-three-shard-interop
```

输出目录必须不存在。生成器仅从可检查的 JSON 自建合成 pickle,以
`--device 0 --events-per-slice 1 --no-focus` 分别调用真实 `pt-snap import`:
`msinsight`(compatibility-v1,内联调用栈)与 `pt-snap-native-v2`(interned 调用栈)。
不打开 GUI、不运行原版生产端。已提交的 v1 产物在只读测试打开前校验 checksum;
新生成结果复现物理语义,不承诺含时间/路径/版本差异的逐字节相等。

回归检查事件唯一归属、全部九个物理 event 字段、完整栈文本和 block 引用。
free-requested 行为 ID 1、action 5、address 4096、size 16、stream 7,
allocated/active/reserved 计数器为 0/16/64 字节(消费端的
Caching Allocated/Active/Reserved 字段)。Block 0 保留
`allocEventId=0`、`freeEventId=2`;后者指向 **completed** free。
现有消费端只查当前片的 predicate 在片 0/2 找不到 event 1。
独立 SQL oracle 先按 manifest 寻找与 `allocEventId < id < freeEventId` 相交的候选片;
`freeEventId < 0` 时查到该设备最后一个真实事件,再按 action/address 与相同边界过滤。
另设 v1/v2 地址复用及 open-ended pending-free/allocated 测试,防止关联后一次生命周期,
并校验应返回唯一事件或无事件。原有九事件 `lifecycle_data` 包含 OOM,故不用作此最小
v1 fixture;既有完整回放覆盖继续保留。

另行获准 GUI 复测时,应使用新生成的原始 `three_shard_free_requested.pkl` 及其邻接
`.msinsight` 目录。不要让会写派生缓存表的消费端修改已提交的只读 fixture。
不宣称 native-v2 是 msinsight GUI 输入。本项仅验证生产端/fixture 契约,
没有修复或验证 [msinsight #625](https://gitcode.com/Ascend/msinsight/issues/625),
也不覆盖下方带日期验收记录中的消费端限制。

## 性能:相同已审小样本,三模式

测量使用 `snapshot_expandable.pkl`,581140字节,SHA256
Expand Down
93 changes: 93 additions & 0 deletions tests/fixtures/interop/PROVENANCE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# Three-shard free-requested interoperability fixture

## Introduction record: 2026-10-11, pt-snap-cli #249

Purpose: an independently authored, minimal producer-side regression for
[pt-snap-cli #249](https://github.com/codeasier/pt-snap-cli/issues/249) and a
retest input for [msinsight #625](https://gitcode.com/Ascend/msinsight/issues/625).
This record does **not** establish a consumer fix or a successful GUI run.

### Source, rights and privacy

- `three_shard_free_requested.json` is new synthetic data authored for this
repository, under its MIT license. It contains one device, one 64-byte segment,
one 16-byte block, and exactly three events: alloc 0, free_requested 1,
free_completed 2. Stream 7 and address 4096 are invented constants.
- Frame filenames/functions are invented. There are no user captures, model
sessions, account details, host paths from a user machine, or vendor code.
Nothing was copied from or run in msinsight. SQL predicates/schema are
independently expressed interface facts; the fixed target remains
Ascend/msinsight `101f65b877a267ffd5f66ea3834706057ba243e5`.
- The committed artifacts are inspectable JSON/Python and SQLite, **not pickle**.
The executable-snapshot acceptance records in `../snapshots/` are unchanged.
Their historical approvals do not approve new pickle inputs.

### Generation and reproducibility boundary

The generator used pt-snap-cli source commit
`55b5981fc03df93a60b809153e8abee5f980ed29`, Python 3.12.14 and SQLite 3.53.1.
It created a new primitive-only protocol-4 pickle from the adjacent JSON in
`/tmp/pt-snap-249-reviewed-fixture/`, then called the actual CLI subprocess:

```bash
PYTHONPATH="$PWD/src" python tests/fixtures/interop/generate_three_shard.py \
--output /tmp/pt-snap-249-reviewed-fixture
```

The directory must be absent. No external pickle is read by the generator;
imports deserialize only this locally constructed synthetic source. The stored
pickle for that generation was 695 bytes, SHA256
`3c9513c7a8dca98d589cd82fc90ef7e177eec70b57887a1601b601f418a79f1f`.
Static opcode inspection finds no GLOBAL/STACK_GLOBAL, REDUCE/BUILD,
INST/OBJ/NEWOBJ/NEWOBJ_EX, persistent-ID or extension opcodes. The pickle is not
committed; this observation is not a claim that arbitrary pickles are safe.

The committed `three_shard_free_requested.pkl.msinsight/` is the unmodified
compatibility-v1 output: three 53,248-byte SQLite shards and a 2,174-byte complete
manifest. Its `sourceFile` is the public, disposable generation path above, not
a relocatable GUI entry or a path expected to exist on the reader's machine.
The CLI timestamp and importer version remain the actual recorded values.
The native-v2 export is generated and tested but not duplicated in the repository.

Reproduction means equal physical real events, lifecycle references, ownership,
and manifest ranges, verified against explicit expectations in
`tests/test_interop_three_shard.py`. It is not a cross-Python/SQLite byte-for-byte
reproducible-build claim. New generation timestamps, source locations, versions,
SQLite serialization and therefore artifact hashes may differ. Never normalize
or rewrite shipped metadata to hide those differences. `checksums.json` records
exact bytes and SHA256 for the recorded JSON, generator, manifest and every DB;
tests verify them before opening the committed databases and check read-only
inspection leaves them unchanged. Changing an input/artifact requires review and
an explicit checksum/provenance update; do not refresh checksums to silence a
regression. Preserve this introduction record when recording future changes.

### Expectations and limitations

- All real event rows appear once, only in their owning shard. Negative synthetic
boundary rows are separate and must not be counted as real events.
- The free-requested row is `(id=1, action=5, address=4096, size=16, stream=7,
allocated=0, active=16, reserved=64)` and has the complete two-frame text stack.
These three counters correspond to the consumer's Caching Allocated/Active/
Reserved fields, in bytes. No reconstructed ordered-frame extension is claimed.
- Block 0 has `allocEventId=0` and `freeEventId=2` in every shard. The latter refers
to **free_completed**, never free_requested. Its allocation and completed-free
reference text is preserved across shards.
- A current-shard-only free-requested lookup misses event 1 when viewing shard 0
or 2. Candidate discovery intersects manifest ranges with
`allocEventId < id < freeEventId`; for `freeEventId < 0`, it includes the
device's final real ID. Then action/address and the same bounds select rows.
- Address reuse and open-ended pending-free/allocated lifetimes are separate
generated tests, not extra events hidden in the dedicated three-shard artifact.
The older nine-event `lifecycle_data` includes OOM and cannot be this minimal
v1 fixture. Existing full-replay event coverage remains useful and unchanged.
- Only pt-snap producer/SQL contracts are tested here. The msinsight GUI/C++ fix,
original-producer execution and #625 consumer acceptance remain separate work.

## Portability follow-up: 2026-10-11

Checksum member keys use POSIX separators on every host. Narrow Git attributes
pin the recorded JSON/Python/manifest text to LF and SQLite members to binary,
so `core.autocrlf=true` does not change their accepted bytes. A temporary-repository
checkout regression checks actual Git attributes and byte preservation alongside
an unprotected CRLF-converted control. No recorded artifact bytes or checksums
were changed for this portability correction.
43 changes: 43 additions & 0 deletions tests/fixtures/interop/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Three-shard free-requested fixture

[Provenance and exact generation identity](PROVENANCE.md) ·
[File sizes and SHA256](checksums.json)

The committed compatibility-v1 dataset is
`three_shard_free_requested.pkl.msinsight/manifest.json`. Device 0 has exactly
three one-event shards: allocation 0, free-requested 1, completed-free 2.
Read it without changing its bytes. The source is synthetic JSON; no executable
pickle is committed.

Run the producer checks from the repository root:

```bash
PYTHONPATH="$PWD/src" python -m pytest tests/test_fixture_provenance.py
PYTHONPATH="$PWD/src" python -m pytest tests/test_interop_three_shard.py
```

To prepare a new isolated bundle for a separately authorized consumer retest:

```bash
PYTHONPATH="$PWD/src" python tests/fixtures/interop/generate_three_shard.py \
--output /tmp/NEW-three-shard-interop
```

Use an absent destination. This generates a new local synthetic pickle and both
real CLI exports, with `--device 0 --events-per-slice 1 --no-focus`; it uses no
network, GPU/NPU, model sessions or original producer. The generator does not
launch a GUI and will not replace an existing output directory.

The GUI's original-source entry requires the **generated**
`three_shard_free_requested.pkl`, adjacent to its generated `.msinsight` directory.
Do not open the committed fixture directly in a consumer that modifies its DBs,
and do not transplant its manifest over another cache. Use the new bundle for
that separate run. Native-v2 output is for pt-snap's reader; it is not a claimed
msinsight input. Record consumer revision and actual observed Free Requested
panel behavior for all three shards before claiming a fix.

The tests demonstrate the current-shard-only lookup limitation and independently
verify bounded manifest-wide association, including separate address reuse and
open-ended cases in both layouts. See the bilingual
[English acceptance guide](../../../docs/en/interop-acceptance.md#three-shard-free-requested-fixture)
and [Chinese acceptance guide](../../../docs/zh/interop-acceptance.md#三片-free-requested-fixture).
26 changes: 26 additions & 0 deletions tests/fixtures/interop/checksums.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
{
"generate_three_shard.py": {
"bytes": 2462,
"sha256": "b484f505883786d5dec296e97ba20eaba6409628432ed68d263e6afcc1032833"
},
"three_shard_free_requested.json": {
"bytes": 1667,
"sha256": "1986dbcbe103c84ee209e26be68504c6d7ef6aef7cc5bbf95bdfed0bf57c4eaa"
},
"three_shard_free_requested.pkl.msinsight/device_0/slice_00000.db": {
"bytes": 53248,
"sha256": "51bc6b5892c2118d9501744a983bd8fe3733446b8ac064d378bfbf3890f240ef"
},
"three_shard_free_requested.pkl.msinsight/device_0/slice_00001.db": {
"bytes": 53248,
"sha256": "a6afbd48523c489452e1f6e2c6d8eaba4af655a686cbc9343c4ccb56a497c82b"
},
"three_shard_free_requested.pkl.msinsight/device_0/slice_00002.db": {
"bytes": 53248,
"sha256": "58743c54ff808ee66a461f4e0b78ab60ff020e8291e275e98e2bfcd2877e61f6"
},
"three_shard_free_requested.pkl.msinsight/manifest.json": {
"bytes": 2174,
"sha256": "c71b91d3792c82f53379d31bc3699ebab38c4620b31ea1678bfb1b55f3eb61fa"
}
}
82 changes: 82 additions & 0 deletions tests/fixtures/interop/generate_three_shard.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
"""Generate new CLI exports from the adjacent inspectable synthetic JSON only.

No external pickle, original producer, GUI, network or hardware is used. The
pickle is built locally from primitive JSON values, not supplied by a third party.
"""

from __future__ import annotations

import argparse
import json
import os
import pickle
import subprocess
import sys
from pathlib import Path

SOURCE = Path(__file__).with_name("three_shard_free_requested.json")
FORMATS = ("msinsight", "pt-snap-native-v2")


def import_source(source: Path, output_format: str, home: Path) -> dict:
"""Exercise the real CLI entry point in a subprocess with isolated focus."""
env = os.environ.copy()
env.pop("PT_SNAP_DB_PATH", None)
env["HOME"] = str(home)
env["XDG_CONFIG_HOME"] = str(home / ".config")
result = subprocess.run(
[
sys.executable,
"-m",
"pt_snap_cli.cli",
"import",
str(source),
"--format",
output_format,
"--events-per-slice",
"1",
"--device",
"0",
"--no-focus",
"--json",
],
cwd=home,
env=env,
capture_output=True,
text=True,
check=False,
timeout=60,
)
if result.returncode:
raise RuntimeError(
f"CLI import failed ({result.returncode}): {result.stderr}{result.stdout}"
)
return json.loads(result.stdout)


def write_source(directory: Path, data: dict) -> Path:
"""Serialize only caller-constructed synthetic data to a NEW local file."""
path = directory / "three_shard_free_requested.pkl"
with path.open("xb") as stream:
pickle.dump(data, stream, protocol=4)
return path


def generate(output: Path) -> dict:
"""Keep the generated source adjacent to its v1 GUI cache; never replace."""
output = output.absolute()
output.mkdir()
source = write_source(output, json.loads(SOURCE.read_text(encoding="utf-8")))
exports = {fmt: import_source(source, fmt, output) for fmt in FORMATS}
return {"source": str(source), "exports": exports}


def main() -> None:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--output", type=Path, required=True, help="New, absent output directory")
args = parser.parse_args()
print(json.dumps(generate(args.output), indent=2))


if __name__ == "__main__":
main()
Loading
Loading