CI: boundary-oracle mutation ratchet (re-land, ANSI fix + diagnostics) - #18
Merged
Conversation
…roduce Adds captured-output dumping to the mutation harness for VACUOUS/INVALID verdicts (the black box that made the first re-land's CI failure un-diagnosable), and re-adds the CI step in temporary full-mode-on-every-event so a PR reproduces the push-to-main failure with that output visible. Not a keeper — once the 7x-pool-boot incompatibility is understood, this reverts to the gated form. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
mnemion | 09ef7b3 | Jun 25 2026, 07:08 PM |
The diagnostic dump proved the mutations DID work in CI — the oracles caught every
break (context-capability AssertionError "expected undefined to be true";
sql-ident "expected [Function] to throw"). The harness misread its own output:
CI forces ANSI color, which interleaves escape codes into vitest's summary
("Tests \x1b[..m1 failed"), so the `Tests \d+ failed` regex missed and every
ANCHORED edge was misfiled INVALID. The 7x-pool-boot theory was wrong; all boots
ran fine.
Fix: run vitest with NO_COLOR/FORCE_COLOR=0, strip ANSI before matching, and detect
ANCHORED via any of vitest's failure markers (summary line, "Failed Tests N", or a
"FAIL ….test." line) rather than one brittle pattern.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Full-mode now verified green in CI (prior commit's diagnostic run: 6 anchored, 1 shadowed, 0 invalid). Switch the step from diagnostic full-mode-on-every-event back to the intended gated form: incremental on PRs, full backstop on push to main. Keep the captured-output dump — it's what made this fixable. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
daniloc
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Re-lands the mutation ratchet as a CI gate after #16's push-to-main failure, with the actual root cause fixed.
Root cause of the first failure
Not the 7×-pool-boot I guessed in the revert. The mutations worked in CI — every oracle caught its break (e.g. context-capability
AssertionError: expected undefined to be true; sql-identexpected [Function] to throw). The harness misread its own output: CI forces ANSI color, which interleaves escape codes into vitest'sTests N failedsummary, so the ANCHORED regex missed and every passing edge was misfiled INVALID. Confirmed via a diagnostic run on this branch that dumps captured oracle output.Fixes
NO_COLOR, strip ANSI before matching, and detect ANCHORED via any of vitest's failure markers.--changedvs base (incremental); push to main → full backstop. Full-mode verified green in CI on this branch (6 anchored · 1 shadowed · 0 invalid) before re-gating.🤖 Generated with Claude Code