Skip to content

test: pinned release E2E for installed HTML and msinsight consumer - #264

Closed
codeasier wants to merge 6 commits into
mainfrom
test/release-e2e-07bdf66-20261012
Closed

codeasier wants to merge 6 commits into
mainfrom
test/release-e2e-07bdf66-20261012

Conversation

@codeasier

@codeasier codeasier commented Oct 12, 2026 •

Copy link
Copy Markdown
Owner

Evidence correction: the initial two hosted runs passed functional assertions but failed screenshot privacy inspection. Do not redistribute their PNG artifacts. Those artifacts require user cleanup; current code keeps real-fixture images private and publishes only invented-label proofs plus numeric summaries.

Scope

Tests and read-only acceptance workflows only. No product-source changes, publishing, tags, deployment, or automatic merge.

Product candidate is pinned to 07bdf660f10f03a9c13ab313fae149adf62355f8. Both hosted jobs build and audit that exact source separately from the reviewed test harness. Hosted artifact hashes are recorded independently; source equivalence does not imply identical wheel bytes.

Installed HTML gate

  • Clean offline wheel installation outside the source checkout, isolated HOME/CWD.
  • Actual installed CLI imports the reviewed fixture and emits HTML.
  • Independent SQLite lifetime/frame oracle checks every rendered node, bytes, counts and proportions.
  • Chromium exercises search, native keyboard navigation, deep focus, parent/reset, duplicate and HTML-like labels, tooltips and Back restoration.
  • Upload allowlist: numeric summary and two explicitly named screenshots from a separate invented-label privacy regression. All real-fixture screenshots, HTML, paths, callstacks, databases, command output and tracebacks stay runner-private.

Optional pinned msinsight consumer gate

  • Restores original msinsight 101f65b877a267ffd5f66ea3834706057ba243e5 with pinned official dependencies; no upstream edits.
  • Separately audited installed wheel exports a five-shard compatible artifact.
  • Actual GUI import and window navigation 1→2→1→4, peak marker 7430, then 167 protocol assertions.
  • Requires no-parser-reexecution trace evidence, original artifact preservation, and logical equality after allowed consumer cache creation.
  • Uploads only sanitized summary, never raw traces/DBs/screenshots.

Validation before opening

  • New harness contracts: 31 passed together.
  • HTML related governance/audit tests: 121 passed before integration.
  • Ruff, Black and diff checks pass; independent review completed.
  • Local installed CLI producer and SQL oracle pass. Local installed-browser GUI remains unexecuted because this environment cannot launch/reach the required browser; this PR provides a supported hosted execution path.
  • Fresh separate source-browser gate: 12 browser tests plus 7 provenance tests passed. This is not a substitute for the installed-HTML gate.

Hosted acceptance results are pending. Keep this PR draft until the full release-validation report is reviewed. Final release version/changelog selection is outside this test-only change.

codeasier commented Oct 12, 2026 •

Copy link
Copy Markdown
Owner Author

Correction at 2026-10-12 05:27 UTC:

  • Installed-HTML functional assertions passed, but subsequent image-by-image inspection found screenshot redaction failure in the empty-results state. The screenshot privacy gate is FAILED. Do not use or redistribute the screenshot artifacts from runs38207610426 or38207805588. The earlier safe-artifact link has been removed. Removal/replacement is pending the required authorization.
  • A tests-only fix is in progress. Functional PASS does not imply safe evidence publication or overall release approval.
  • Product remains pinned to07bdf660f10f03a9c13ab313fae149adf62355f8; no production-source change.
  • Initial MS workflow validation issue was fixed in c3ed8eb. Official actionlint validates both workflows, and 29 new harness contracts pass. MS hosted acceptance and full release matrix are still pending.

PR remains draft.

Copy link
Copy Markdown
Owner Author

Privacy correction verified at head 31ea0078ece3ae4a5597cee463a0b64992fba144:

  • Current installed-HTML job passed all actual installed-CLI/browser assertions, with zero console/page errors.
  • Public artifact11728275962 contains exactly summary.json, privacy-synthetic-initial.png, and privacy-synthetic-empty.png. Both invented-label proof images were inspected, including focused/scrolled empty-results state; sensitive-label surfaces are hidden.
  • Real-fixture screenshots remain runner-private and are not uploaded. Synthetic proof images are not evidence screenshots of the real fixture UI; the real fixture UI is verified by executable assertions and numeric summary.
  • Earlier artifacts11727718442 and11728405253 remain excluded from evidence and require user cleanup. Their privacy failure is not retroactively cleared by this fix.
  • Combined new contracts:31 passed; actionlint passes. Product source stays07bdf660. Other release gates are still running; no overall release approval.

Copy link
Copy Markdown
Owner Author

CI follow-up at head 34dfa3b7650cc5e778d127bf82f2ae7ef3fe1492:

The first full compatibility runs exposed one new-harness governance failure: a literal legacy source-folder name in the external-consumer staging code. Earlier local checks missed newly untracked files because this governance gate intentionally scans Git's tracked manifest.

Fixed by functional discovery of the sole pinned parser entrypoint, with missing/duplicate/layout/symlink rejection. The governance allowlist and product source are unchanged. All13 new files were staged before rerunning validation: 64 governance + new harness tests passed, including37 new contracts; Ruff/Black/actionlint pass. Symlink guard unit tests are Windows-neutral, and actual Linux symlink negatives were also checked. Local staged tree exactly matches the pushed tree.

Hosted MS GUI/protocol/reuse/preservation already passed at both earlier harness heads; final-head CI is running again. This is a test harness correction, not a product-source fix. Real-fixture screenshots remain private; old privacy-failed artifacts remain excluded.

Copy link
Copy Markdown
Owner Author

验证更新(2026-10-12 06:40 UTC),测试 head 5a651ef0408904212148837342b74a498cfcf988:

  • 本次只补充 msinsight GUI 的固定阶段标签和有界数值诊断,未改浏览器操作、断言、超时或产品源码。独立 AST 对照通过;本地 57 个新增 harness tests + 27 个 governance tests 全部通过。
  • 当前 Test run 必需作业通过。Python 3.10/3.11/3.12 均完成 2,977 个精确测试身份核对;3.13 为 2,979 passed、14 expected skips、7 nightly deselected,综合覆盖率 93.19%。nightly 另行执行,不把条件跳过视作通过。
  • 当前 msinsight run 通过实际 GUI import、窗口 1→2→1→4、167 项 protocol assertions、复用和数据保留。安全摘要显示 phase=complete、零 page errors。Artifact 11729597557 仅含 summary.json;ZIP SHA-256 2c2d7594feff91ce8dccef508e7183d85a31a9510884f0993bcb430ea5a141ed。
  • 必须保留可靠性限制:前一 head 的 run 38209513537 曾在 GUI navigation 失败,只有安全摘要存留,具体原因仍未确定。当前通过不能证明该偶发失败已被修复;诊断补丁没有修改或修复已确认的产品原因。
  • 当前 installed HTML run 通过。公开 artifact 11729324943 严格只有 summary.json 与两张 invented-label privacy proof;真实 fixture PNG 不发布。旧 artifact 的清理事项仍按前述警示保留。

产品候选仍为 07bdf660f10f03a9c13ab313fae149adf62355f8。本地 canonical wheel 与 hosted rebuilt wheel 是不同 SHA 的产物,未混同证据。大 fixture 全数据校验、nightly 及最新 Platform 身份审计尚未全部终结;PR 继续保持草稿。

Copy link
Copy Markdown
Owner Author

候选版本验收完成(2026-10-12,按批准的分层抽样范围)

产品候选固定为 07bdf660f10f03a9c13ab313fae149adf62355f8,本 PR 仍是仅新增 13 个测试/workflow 文件的草稿,head 5a651ef0408904212148837342b74a498cfcf988。没有修改产品源码、合入、打 tag 或发布。

已完成证据

  • 原候选 Python 3.13:2922 passed + 2 个逐 ID 核对的预期 skip;综合 statement+branch 覆盖率 93.19%。7 个 nightly 全部通过;200 seeds × 2 formats 的 400 个差分用例全部通过,28,864 次比较。
  • 当前 Test、Platform、installed HTML 和 msinsight 均终态 SUCCESS。Linux compatibility / Windows 分片并集 / macOS 的精确身份与 skip 均独立审计;57 个新增 harness tests 各平台通过。PR 条件跳过的 dataset job 不算通过,已由独立 nightly 验证补足。
  • 七份真实 fixture 的本地 installed CLI/API 矩阵终态:1024 个唯一要求 = 1008 PASS + 12 个明确的默认预算拒绝 + 4 个经批准的穷举转抽样替代;没有未解决、缺失或未知状态。312 个 catalog cells = 274 个支持查询结果通过 + 32 个显式 unsupported-contract 拒绝 + 6 个正确预算拒绝。1747 条完成命令的 JSON/stdout/stderr 审计是另一个分母,不与测试数相加。
  • 131k 三种格式的事件/生命周期穷举完成;628k standalone 生命周期 213,725 个分配穷举完成。628k 两种 dataset 格式按批准的固定种子 20261012 分层验收:每格式 28 个事件窗口、156 个唯一事件 ID、9 个生命周期点,覆盖 20 个分片、19 对边界、峰值、关键动作和 5 个内部随机分层。两格式共 69 次新 API 查询、6 次新 CLI 调用;compatibility 的 5 个生命周期点与 2 次历史 CLI 证据单列复用。
  • 原 compatibility 穷举已按范围调整停止:只认可完成独立校验的 141 页、allocation-event ID 0–288767、97,688 个生命周期。状态为 STOPPED_BY_SCOPE_CHANGE,不冒称完整穷举。最后未持久证明父进程比较完成的响应不计入通过覆盖。
  • 七个原始 fixture、canonical wheel 和全部 428 个 tracked source blobs(含 243 个 tests)再次核验一致;无源数据改动。

必须保留的限制

  • 大范围查询在默认 64 MiB work budget 下可能正确拒绝且不返回部分全局结果。完整 leak 请求的拒绝与独立验证的 min_size=0, limit=50 正例分开;指定 midpoint 的 256 MiB opt-in 结果不代表默认查询成功。
  • 原主矩阵 in-process API 的 HOME/env 隔离未被明确证明;其已安装包来源与 CLI clean-env 证据保留,后续 clean reruns 只覆盖其明确声明的范围。采样不是穷举,compact event 输出也不替代独立 full-stack/frame 检查。
  • 当前 msinsight GUI 与 167 项 protocol assertions 通过;此前 run 38209513537 的 GUI navigation 失败仍原因未明。诊断补丁没有被宣称修复了已确认的产品原因。
  • 旧 privacy-affected artifacts 11727718442 / 11728405253 在 10:39 UTC 复核时仍存在,须由 owner 清理,禁止再次分发。当前 HTML artifact 仅含安全摘要与 invented-label privacy proof;真实 fixture PNG 不发布。

0.4.1 发布准备

0.4.1 未打 tag 的预演 wheel/sdist、来源/assets/license/metadata 审计、实际小 fixture 离线安装验收通过。wheel SHA-256 17f0d4cdf8bf711be07b1e498635f827fea35374c7822d90cfc1bc4f50c0db69,独立 sdist 重建逐字节一致;与 dev canonical wheel 相比仅 METADATA Version 和 RECORD 不同,117 个其他成员相同。广泛 dev-wheel 验收没有被重新标作执行了 0.4.1 bytes。

正式发布仍未放行:源码缺少 0.4.1 CHANGELOG 条目(发布 workflow 会拒绝),提案尚未应用/批准,最终 tagged-release 实际 bytes 仍需绑定和适用门禁,且旧 artifact 清理待完成。PR 保持草稿,以上是候选验收结果而非发布批准。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant