Skip to content

chore(testsuite): re-pin to latest (193e551) + monthly re-pin workflow (kiln#360)#361

Merged
avrabe merged 1 commit into
mainfrom
chore/testsuite-repin-latest
Jun 24, 2026
Merged

chore(testsuite): re-pin to latest (193e551) + monthly re-pin workflow (kiln#360)#361
avrabe merged 1 commit into
mainfrom
chore/testsuite-repin-latest

Conversation

@avrabe

@avrabe avrabe commented Jun 24, 2026

Copy link
Copy Markdown
Collaborator

Resolves the kiln#360 confound and establishes the "regularly re-pin" practice.

The confound (kiln#360)

external/testsuite was unpinned: gitlink recorded c337f0d while the working tree had 7e0b83a checked out. So a fresh git worktree/CI checkout tested a different suite than the main worktree — which looked like ±18-assertion runner non-determinism but was a submodule-version mismatch. Same-commit runs are deterministic (verified: the cast cluster and full-suite non-type-subtyping results are identical across runs).

What this does

  • Pins the gitlink to WebAssembly/testsuite @193e551 (latest, 2026-06-17), matching the checkout → conformance is now reproducible across CI/worktrees.
  • Adds .github/workflows/testsuite-repin.yml — monthly (+ workflow_dispatch) job that bumps to latest upstream, re-runs WAST conformance, and opens a PR for review of the new baseline (spec evolution can add failing files, so it's reviewed, not auto-merged). This is the "regularly re-pin" cadence while keeping it pinned at all times.
  • Updates the conformance note in architecture-components.yaml with the real, reproducible baseline.

New baseline (measured from source @193e551)

263/281 files pass; 65980 assertions pass / 529 fail. Bumping 7e0b83a → 193e551 added +184 passing assertions and +1 passing file with zero new failures — the spec evolution is strictly more coverage that kiln handles. Remaining 17 failing files dominated by the GC custom-descriptors proposal + br_on_cast/return_call_ref (Issue 149).

Note

PRs opened by the scheduled job use the default GITHUB_TOKEN, which doesn't re-trigger CI on the auto-PR — a human reviewer re-runs CI when reviewing the new baseline (acceptable, since the baseline needs human review anyway).

rivet validate: 0 errors. Refs kiln#360, #149.

🤖 Generated with Claude Code

…thly re-pin workflow

Resolves the kiln#360 confound: the submodule was unpinned (gitlink c337f0d ≠
checked-out 7e0b83a), so fresh worktrees/CI silently tested a different suite —
which looked like runner non-determinism but was a submodule-version mismatch.
Same-commit runs are deterministic.

- Pins the gitlink to WebAssembly/testsuite @193e551 (latest, 2026-06-17), matching
  the checkout, so conformance is reproducible across CI/worktrees.
- Adds .github/workflows/testsuite-repin.yml: monthly (+ manual) job that bumps to
  latest upstream, re-measures WAST conformance, and opens a PR for review of the
  new baseline (spec evolution can add failing files → reviewed, not auto-merged).
- Updates the conformance note: 263/281 files pass, 65980 assertions pass / 529 fail
  @193e551 (vs the stale '332/74 @7e0b83a'). Bumping 7e0b83a→193e551 added +184
  passing assertions and +1 passing file with ZERO new failures.

rivet validate: 0 errors.
@github-actions

Copy link
Copy Markdown

🔍 Build Diagnostics Report

Summary

Metric Base Branch This PR Change
Errors 0 0 0
Warnings 5 5 0

🎯 Impact Analysis

Issues in Files You Modified

  • 0 new errors introduced by your changes
  • 0 new warnings introduced by your changes
  • 0 total errors in modified files
  • 0 total warnings in modified files
  • 0 files you modified

Cascading Issues (Your Changes Breaking Other Files)

  • 0 new errors in unchanged files
  • 0 new warnings in unchanged files
  • 0 unchanged files now affected

Note: "Cascading issues" are errors in files you didn't modify, caused by your changes (e.g., breaking API changes, dependency issues).

✅ No Issues Detected

Perfect! Your changes don't introduce any new errors or warnings, and don't break any existing code.


📊 Full diagnostic data available in workflow artifacts

🔧 To reproduce locally:

# Install cargo-kiln
cargo install --path cargo-kiln

# Analyze your changes
cargo-kiln build --output json --filter-severity error
cargo-kiln check --output json --filter-severity warning

@codecov

codecov Bot commented Jun 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@avrabe avrabe merged commit bc3854c into main Jun 24, 2026
18 checks passed
@avrabe avrabe deleted the chore/testsuite-repin-latest branch June 24, 2026 20:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant