Skip to content

ci: tier compatibility tests and require nightly wheel acceptance - #255

Merged
codeasier merged 2 commits into
mainfrom
codex/248-ci-tiers
Oct 11, 2026
Merged

codeasier merged 2 commits into
mainfrom
codex/248-ci-tiers

Conversation

@codeasier

Copy link
Copy Markdown
Owner

Summary

  • Run Python 3.10–3.12 compatibility tests with -m "not slow"; keep the existing full slow suite, branch coverage, and unchanged 90/90/88/84 coverage floors on Python 3.13.
  • Mark the existing million-row deep-offset pagination cases slow, leaving their small semantic cases in every Python lane.
  • Add nightly (03:17 UTC) and manual installed-wheel dataset acceptance to the existing Test workflow. Build and audit the wheel before invoking the existing public CLI/API acceptance node.
  • Fail closed on missing wheels, zero/extra collection, unexpected skips, failed tests, missing JUnit, or mismatched wheel evidence. Upload logs, JUnit and a checkout-SHA/wheel-SHA summary, including failed setup evidence.
  • Pin PR checkout to the published PR head SHA. Preserve least-privilege permissions, provenance, lint, browser, packaging and release gates.

Refs #248. This is the CI foundation, not completion of the umbrella issue or #250. The existing installed-wheel test still uses pip --target and 500,002 events; clean venv isolation, two full default-sized shards and broader public CLI coverage belong to the follow-on acceptance PRs.

Validation

  • Local Linux/Python 3.12: 75 focused provenance/governance/workflow/wheel-audit/runner tests passed; 21 runner/wiring tests rechecked after the fail-closed setup correction.
  • Repository Ruff and Black checks passed; basedPyright: 0 errors (existing warnings remain); git diff --check passed.
  • Built and audited the wheel from local commit c5e2d0950208e79d6704557dcb3b4808f30aba2f. The required installed-wheel runner passed exactly one test with matching checkout and wheel evidence.
  • Final full compatibility and coverage validation remains pending. An earlier in-flight local compatibility run was invalidated by concurrent test/metadata edits and is not claimed as passing.
  • Exact published-head GitHub checks and a manual/nightly run have not yet been exercised. A PR-only green run is not evidence that the installed-wheel nightly gate ran. Manual dispatch availability must be verified against the published ref; GitHub's default-branch/registered-workflow requirements may block dispatch before merge.

Follow-on tier integration

New high-cost #250/#252 acceptance cases should be explicitly marked nightly and selected only by the nightly/manual tier in a follow-on integration change. Existing slow coverage must remain in the Python 3.13 PR lane. No live model, real user session, upstream GUI or local Mac was used.

Copy link
Copy Markdown
Owner Author

Follow-up tier integration is now published at 68d89670b188c93e41b1a5d8f41903057e5ada06 (reviewed tree 83c4403f055eb4caed4d3de63b5b56fb87192ad1).

  • Register nightly; Python 3.10–3.12 select not slow and not nightly, Python 3.13 selects not nightly with unchanged coverage floors. All existing non-nightly slow regressions remain in 3.13.
  • The nightly runner now executes all nightly cases, requires the existing installed-wheel node exactly once, rejects missing configured nodes, duplicate identities and every failed/error/skipped case, and records complete executed identities.
  • Local exact-tree verification: 86 focused checks plus Ruff/Black passed; built/audited wheel nightly gate passed 1 test with no skips. Wheel SHA256 b981fc1a144920cd153a25919bd11c48f374a723a0f6745b4b6664361c93d49c. Serial full PR-coverage validation remains running.
  • Previous head 430acf97 completed all 7 applicable PR jobs successfully; the nightly job was intentionally skipped and is not counted as executed acceptance. New-head CI is pending.

Future costly cases add their required node identities alongside their actual files. No absent future tests are required by this commit. GitHub manual/nightly execution is still unverified; local acceptance does not replace that evidence.

Copy link
Copy Markdown
Owner Author

Validation update for exact head 68d89670b188c93e41b1a5d8f41903057e5ada06: Test run 38145468881 completed successfully. All 7 applicable PR jobs passed; the nightly-only job was intentionally skipped.

Python 3.13: 2,652 passed, 14 skipped, 1 nightly deselected, branch coverage 93.18%; independent coverage floors, wheel build/source audit, clean-wheel acceptance, and required Chromium job passed. The 14 regular-suite skips are the 12 browser cases covered by the separate required Chromium job and 2 unsupported slice-leak combinations.

Frozen local tree also passed the full PR coverage run (2,652/14/1; 93.18%) and the real required nightly wheel runner (1 passed, no skips). Remote manual/nightly execution remains unverified: the available connector has no dispatch action, local gh is unauthenticated, and the assistant-cloud GitHub browser is signed out. No assertion is made that the nightly workflow has run remotely. Draft state retained.

codeasier commented Oct 11, 2026 •

Copy link
Copy Markdown
Owner Author

Local combined verification of #255–#263

A separate local integration branch combined the nine independently reviewed changes, including the bilingual documentation correction now on #260. No GitHub merge was performed.

Frozen local source 3cac4514e81b8fa0c746bf152ed7a9dbc0bd6227, tree f1b88fdf580667a09fcf3fa52e42c6f748947b05; audited universal wheel SHA-256 d8565792fa00a53b0b0ef55a0c2b997533159f801c20ce8974d55198882ae992.

  • Python 3.13 full PR tier: 2,909 passed, 14 skipped, 7 nightly deselected, 93.19% combined statement-and-branch coverage (branch accounting enabled); all critical-module floors passed.
  • Focused combined guards: 290 passed; Ruff, Black, provenance and wheel/source audit passed; type check 0 errors (existing warnings retained).
  • Python 3.12 actual generic acceptance: 7/7 passed, zero skips. Includes isolated installed-wheel 1,000,000-event and 500,002-event workloads, three scale/route cases and two resource gates. Runner verified exact source/wheel identity.
  • Python 3.13 exact 200-seed paired differential: 400/400 passed, 28,864 comparisons, 2,128 scratch builds; 11 supported templates, zero unexpected errors.
  • Python 3.12 isolated offline CLI/skills acceptance: both formats passed with two physical shards and 8,180 events each; no live model/agent run.

Environment limits remain explicit: local Python 3.13 lacked ensurepip for the fresh-venv gate, so that unchanged wheel gate was rerun successfully under 3.12; an initial inside-checkout temporary directory was rejected by the isolation guard and corrected without source changes. An extra local Chromium attempt was blocked before page execution by the environment's socket permission restriction. Individual hosted PR browser jobs passed independently. Failed setup logs were retained, not counted as successful tests.

Remote nightly was not dispatched. Windows #257's CI-only two-way partition correction is still being verified and is not included in this frozen source identity. Draft PRs and issues remain open.

Copy link
Copy Markdown
Owner Author

检视结论:approve

对照 #248 CI 分层:3.10–3.12 -m "not slow"、3.13 全量+覆盖率、nightly wheel runner fail-closed,符合约定。test.yml / run_dataset_acceptance.py 核对通过;定向 54 passed。

建议作为合入地基先合。本评论不 approve/merge。

@codeasier codeasier left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM,已转为正式 PR。GitHub 不允许作者账号提交 APPROVE,以本评论 + 此前检视评论作为通过记录。建议作为合入地基先合。

@codeasier
codeasier merged commit e6fb742 into main Oct 11, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant