Skip to content

test: bound dataset validation resources and verify report hash and completion contracts - #263

Merged
codeasier merged 2 commits into
mainfrom
test/253-resource-acceptance
Oct 11, 2026
Merged

codeasier merged 2 commits into
mainfrom
test/253-resource-acceptance

Conversation

@codeasier

Copy link
Copy Markdown
Owner

Summary

Adds acceptance coverage for #253 items 1–4 on top of #261's query-generation guard and #248's nightly runner. This is a narrow resource/test delta: production source and the reviewed query-guard behavior are unchanged.

  • Exercise the real console entry point in fresh subprocesses for compatibility-v1 and native-v2 reports. Count actual hash calls and logical bytes returned by the real hasher. Default cold nonempty reports retain four full hash passes, including both final content guards; immutable is explicitly selected and keeps its existing layout/platform policy.
  • Reject report results when an unselected device changes after either peak selection or attribution, even with hidden generation tokens. Default strong guards from [perf] 默认模式下 report / 解析复用会多次完整哈希整个数据集 #238 remain intact.
  • Run Bash completion with project focus pointing to a dataset directory or manifest; require correct device hints with zero SQLite opens, member-file opens, or hash calls.
  • Bound exporter validation-only memory at 10k and 1m events using separate subprocesses for Python allocation and Linux RSS. Pickle load, replay, fixture construction, conversion, and publication are outside this measurement.
  • Record a fresh-process 500k query on the guarded runtime as trend-only evidence. OS page-cache state is uncontrolled; there is no latency gate.
  • Require both real resource test nodes in the generic nightly runner. Missing, skipped, failed, or errored resource cases cannot produce green acceptance.

References #253, #241, #238. Depends on #261. This does not close the remaining frame-tree, agent, or platform portions of #253, or the signal-cleanup limitations in #252.

RSS measurement correction

The combined run exposed a false failure that an isolated probe did not: Linux ru_maxrss retained the earlier producer/pytest parent's roughly 400 MB high-water across exec. Counting that as validation RSS would be incorrect.

The gate now uses Linux /proc/self/status's kernel-reported current-address-space VmHWM, recording before/after VmRSS separately. Raw ru_maxrss remains a clearly labeled historical diagnostic. There is no RSS sampling thread or sampled-peak claim. The high-water increase is measured relative to the post-import baseline.

The limits are unchanged: Python peak <8 MiB, Python peak growth from 10k to 1m <2 MiB, current-address-space RSS high-water <128 MiB, and RSS high-water increase <32 MiB. New deterministic tests cover inherited-high-water contamination, unit parsing, and unsupported/missing capabilities. A real control retained 160 MiB in the parent: the child's inherited ru_maxrss was 177,471,488 bytes while its current-address-space VmHWM was 25,772,032 bytes. The million-ID Python-list negative control still exceeds the 8 MiB allocation gate.

CI boundaries

The 12 CLI contracts and eight RSS instrumentation tests are unmarked and run on every PR. These two required resource gates are both nightly and slow, reusing the registered #248 marker:

  • tests/core/test_export_validation_resources.py::test_million_event_export_validation_memory_is_bounded
  • tests/core/test_export_validation_resources.py::test_first_500k_dataset_cli_query_startup_trend

The runner also retains its mandatory installed-wheel case, exact wheel hash matching, and complete testcase identities. Evidence includes JUnit properties plus export-validation-memory.json and dataset-cold-start-trend.json.

Resource cases acquire /tmp/ptsnap-large-acceptance.lock themselves. The generic runner is invoked without an outer copy of that lock to avoid nesting deadlock; its cases run serially, and local producer-scale runs were coordinated to avoid overlap.

Verification

Frozen local head: 50f7600b3694cfb94b6d9980330b7d6ccb2c699d

Frozen tree: fae72c92d19e8d28ea028f49dc5d847e1dd956d2

Base tree (query guard + infrastructure): b65254fe5db5b3ff1fb44618f7959b06c7a4986b, identical to #261 head 571fbb7e05349081b673db908bf0f216ab8d200c.

Linux x86_64, Python 3.12.14, SQLite 3.53.1:

  • 190 final focused tests passed, including all 37 query mutation/concurrency/signal cases, report hash guards, completion, resource instrumentation, runner failure guards, and governance.
  • Exact frozen wheel built and audited. SHA-256: a4f0f44dfc461e17226d374ae74c1fcc1a8700712c5f2d587cbb7f70e341d2d9.
  • Generic runner: 3 passed, 2732 deselected, 26.92s; the baseline 500,002-event installed-wheel export/query/API case and both resource gates actually ran. The summary records the matching source commit, wheel hash, and all three required node IDs.
  • Million-event validation Python peak: 8,076 bytes.
  • Separate RSS measurement: VmHWM 25,784,320 bytes, increase 2,453,504 bytes; VmRSS 23,330,816 → 23,932,928 bytes. Historical ru_maxrss 400,400,384 bytes is diagnostic only.
  • First fresh-process 500k CLI query with the final query guard: 1.291923 seconds, recorded without a timing gate or comparison claim.
  • Root Ruff, Black, and diff checks passed. Original failing integrated evidence is preserved separately from the corrected passing run.
  • Independent incremental review approved the corrected frozen tree.

Reproduction (with the intended interpreter and PYTHONPATH=src):

python .github/scripts/run_dataset_acceptance.py --wheel <audited-wheel> --output-dir .tmp/resource-nightly-corrected

No full-suite, paid-model, real-user-session, live-capture, upstream-GUI, or actual macOS/Windows run is claimed here.

Snapshot provenance decision: no-update
Snapshot provenance no-update reason: Only benchmark, test, and acceptance-runner files differ from #261; no snapshot runtime source, rights mapping, or provenance content changes.

Copy link
Copy Markdown
Owner Author

Latest-head hosted CI verified on 861941ea7dc7a165a5602a03ca5e06e24332d1d7: Test run 38147832856 succeeded, with seven applicable jobs passed. Python 3.13: 2718 passed, 14 skipped, 3 deselected, coverage 93.17%; wheel audit and offline clean-wheel CLI/skills acceptance passed. The installed-wheel dataset/nightly job was skipped for this PR event, not executed or passed. Actual local scale evidence is separate; no live-agent/model result is claimed. Remote commit/tree matches the independently reviewed local source.

Copy link
Copy Markdown
Owner Author

检视结论:approve

对照 #253 资源验收:报告 hash 遍数、补全零打开、导出校验内存(VmHWM)、500k 冷启动趋势;nightly 注册 2 resource 节点。含 #261 的 query guard。默认强验证未被削弱。定向 21 passed(跳过 million)。

合入注意:依赖 #261;与 #259/#260 在 REQUIRED_TEST_NODES 硬冲突。请在 #260→#259 之后 rebase,并入全部 REQUIRED + 本 PR 的 2 个 resource 节点,并确认 nightly 最终至少含:wheel 默认规模 + short-final boundary + 3× route million + 2× resource,且 #258 的 200-seed job 仍在。本评论不 approve/merge。

Copy link
Copy Markdown
Owner Author

已核对本轮检视的集成约束,按 #255 → #258 → #261 → #260 → #259 → #263 的依赖顺序处理;#256/#257 可在基础 CI 之后较早集成,#262 放在上述 runner 扩展之后。当前尚未执行合并或 rebase。

合并节点时必须保留并集:两个 clean-wheel 规模/短尾分片节点 + 三个 route-million 节点 + 两个 resource 节点,共 7 个;同时保留 #258 的 200-seed runner 执行步骤(现有实现是同一 nightly job 中独立 step,无需据此改成新 job)。

此前独立本地组合已实跑 7/7 和 400/400 对照,证据见 #255 (comment) 。后续真实集成仍需按每次最新 head 重新校验清单、冲突解决和 CI;不会把旧组合证据当成新提交已通过。

@codeasier
codeasier marked this pull request as ready for review October 11, 2026 16:05

@codeasier codeasier left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM,已转为正式 PR。GitHub 不允许作者账号提交 APPROVE。依赖 #261;请在 #260→#259 之后 rebase 并集全部 REQUIRED + resource 节点。

…e bounds

Rebase reviewed PR263 after PR260 and PR259. Preserve the complete seven-gate union and the independent 200-seed step. Refs #253, #241 and #238.
@codeasier
codeasier force-pushed the test/253-resource-acceptance branch from 861941e to f03d6d2 Compare October 11, 2026 19:48

Copy link
Copy Markdown
Owner Author

已按检视要求在 #260→#259 后 rebase 到实际 main 4556240e40fc7eed62c2fd6eab138907a8877c18。新 head f03d6d2d0b703cc1d36a27947bda779bd9f381d4,树 f518b9df816cce757b96a9d2ff1daa26cea642c0;expected-head lease 更新成功。

两处 runner 冲突采用完整并集:wheel 默认规模 + short-final boundary + 3×route million + 2×resource,共 7 个 REQUIRED;缺失/重复/skip/failure/error 负例保留。200-seed runner 仍是同一 nightly job 内独立 step,四 Windows 分片及 20 分钟/max-parallel 2 不变。生产代码和工作流字节均不变,资源实现沿用原审阅内容。

当前 actual-base 完整树与独立审阅 local preview 9e0ced72a87c109a57b5ccd102755c93ac9aabe0 精确一致;该树91项focused通过。加入后续#262的独立最终预览另实跑7/7 generic与双格式offline,证据属于该最终预览,不标作本远端head已执行。新 head 的完整托管 Test/Platform CI 待验证,合入仍等其成功及当前 main CI 成功。

Use stable slash-separated relative identities in test-owned hash/mutation evidence and test inventory. Add pure Windows/POSIX path regression without changing product behavior or guard assertions.

Copy link
Copy Markdown
Owner Author

托管 Windows 暴露测试证据路径序列化问题,已追加 test-only 修复 9247b98aa20334a68abc4cfb01dd0a962b2d81bf(树 027ec1a30a781f360f986a256a852c3b91c05ea0)。

旧 Platform run 38169563647 的两个已完成 Windows 分片分别仅有1项失败:strong-guard 用例的前置断言已满足(非零退出、content changed、无混代stdout、正确guard次数),但 evidence mutated 为 Windows 反斜杠路径,断言要求 /。旧失败日志保留。

仅修改两个测试文件:hash/mutation evidence 统一 relative_to(...).as_posix(),匹配 inventory 同样使用稳定分隔符;新增 PureWindowsPath/PurePosixPath 回归。所有 guard/error/output/hash-count/byte 断言不变,不增加skip,不改生产逻辑。

3.13 focused/provenance/runner/governance 102 passed;3.12资源/provenance 21 passed。独立审阅在隔离副本上分别以3.12和3.13验证21项通过,并证明替回旧serializer时新的Windows回归确实失败;Ruff/Black/空白检查通过。新 head 完整托管 CI 待验证,尚未合入。

@codeasier
codeasier merged commit 91b06d6 into main Oct 11, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant