Repository navigation
test: expand paired dataset differential acceptance and fix zero-rank completeness - #258
Conversation
|
The previously pending required 200-seed runner has now completed end-to-end on the exact reviewed local tree 400 passed in 577.19s: every expected seed × v1/v2 identity was verified, 28,864 strict comparisons, zero unexpected errors, and 2,128 actual scratch builds. The fail-closed runner validated matching per-case artifacts and wrote status=passed with pytest_exit_code=0. No edits/builds occurred during the run. This is completed local required-runner evidence. Published-head PR CI is still running; remote scheduled/manual execution remains unverified and is not claimed as passing. |
|
Latest-head CI verification (2026-10-11): commit |
|
已处理检视 nit(https://github.com/codeasier/pt-snap-cli/pull/258#issuecomment-6110740929),追加提交
本地 19 项定向/样本溯源 + 60 项真实随机差分测试通过;另有 6 项正则正反例探针通过,Ruff/Black/空白检查通过。远端树与本地审阅树一致;最新提交 CI 正在运行,尚未称为通过。 |
Retain the reviewed independent 200-seed step and evidence upload. No differential runtime, test or runner bytes changed.
|
已与当前 main 主分支要求线性历史,#255–#257 已采用 squash。此次仅解决共享 CI 基础带来的上下文冲突:保留 differential runner 索引、同一 nightly job 内独立的 200-seed step 及其 evidence upload。上述 workflow/runner、差分生产逻辑及测试字节与原审阅 head 完全一致;继承 main 的平台/fixture 变更。 当前集成树本地验证:100 项 runner/CI/provenance/governance 检查 + 88 项实际差分/zero-rank 测试通过,git diff --check 通过。新 head 的托管 Test/Platform CI 待验证;旧 head 的通过记录不作为新 head CI 证据。PR 正文只将自动关闭关键字改为 Refs,议题状态保留。 |
Keep the full non-slow union, twenty-minute job cap and max-parallel two. Update fail-closed partition guards and bilingual reproduction docs; production behavior and differential test cases are unchanged.
|
托管平台集成暴露 Windows 时间预算问题,现追加 CI-only 修复 旧 head 的 Test/Linux 全绿;Platform run 38157189623 的 Windows shard 0 用时 17m43s 成功,shard 1 达到原有 20 分钟上限后取消,日志为 985 passed、8 capability skips、71% 完成,无 assertion failure。旧失败日志保留,不计为通过。 改为四个确定性 Windows 分片,保持 max-parallel=2、每任务 20 分钟、完整 non-slow 集合和警告门禁;macOS 仍完整单任务。不减少种子,不添加 skip,不放宽测试。同步 fail-closed 分片回归及英文/中文说明,生产代码和差分用例字节不变。 本地 52 项 wiring/governance/provenance 测试通过;独立复核 20 项通过,Ruff/Black/空白检查通过。四次真实 collect-only 的相同完整集合共 2,785 项,分片 [697, 696, 696, 696],互斥并集精确相等;40 个默认 seed/format 用例保留,每片 10 项。旧日志重分配的已观测工作约 484/512/512/637 秒,其中旧超时片有未测尾部,不能当作新片完整耗时。新的托管 Test/Platform CI 才验证真实结果;当前待运行。 |
|
最新 head 四个 Windows 任务实际耗时 9m06s / 9m26s / 8m27s / 10m02s,均在原有 20 分钟内;日志对应 691/690/689/688 passed 和 6/6/7/8 capability skips。实际执行身份共 2,785 项、零重复,与独立 non-slow 收集精确相等,40 个 default seed/format 用例均执行。macOS 完整任务成功。 已按授权顺序 squash 合入,main |
Depends on #255; its two infrastructure commits are included in this branch so CI selection works before merging. Review/merge #255 first.
Summary
Refs #251. Built on the CI tier infrastructure from #248.
top_n=0remains conservatively incomplete unlessexact_totalresolves it. Nonzero rank behavior is unchanged. Preserve test: clarify lifecycle contracts across all dataset query paths #254's standalone/dataset leak-scope, unproved-identity and observation-state contracts.Validation
Completed locally:
Pending at draft creation:
--seed-count 200against frozen local commit85250731fc7c6621d0bb6040e09691be7edcbc9f. Its final result is not yet claimed as passed.Reproduce the required runner from the checked-out revision:
Evidence boundaries
These generated fixtures validate reader/query semantics. They do not claim native exporter, live PyTorch/NPU collector, paid-model or real-user-session acceptance. Installed-wheel and producer gates remain separate. No changes are made under
src/pt_snap_cli/snapshot/.