Skip to content

fix: make RegressionRisk detect changed/skipped critical steps (soundness) - #53

Merged
Therealdk8890 merged 1 commit into
mainfrom
fix/regression-risk-soundness
Jul 18, 2026
Merged

fix: make RegressionRisk detect changed/skipped critical steps (soundness)#53
Therealdk8890 merged 1 commit into
mainfrom
fix/regression-risk-soundness

Conversation

@Therealdk8890

Copy link
Copy Markdown
Owner

Parity port of DProvenanceKit (Swift) PR #71. Stacked on #52 (base is fix/deep-review-confirmed-findings, not main) because the fix builds on that branch's alignment_engine.py/CHANGELOG/gate-test changes; GitHub will retarget this to main once #52 merges. The diff here is only the alignment fix.

What & why

An adversarial correctness audit found the RegressionRisk verdict — the signal a CI gate reads — missed the regressions it exists to catch on the strict_audit_v1 profile. The verdict was derived only from removed/reordered alignment states, but a critical step that was materially changed or skipped binds to a same-type event (type match alone clears the matcher threshold), is classified ambiguous, and escaped the verdict entirely. A tampered or skipped critical decision reported RegressionLevel.NONE — even though the equivalence model had already recorded the pair as non-equivalent.

Fix

The verdict now derives from the equivalence outcome. A critical step fires HIGH when it is:

  • removed (0.95),
  • reordered relative to another critical step (1.0) — critical-vs-critical only, so a benign structural/diagnostic step moving past a stationary critical no longer fires a false HIGH,
  • changed beyond equivalence — bound with a differing payload scoring below the profile's semantic_threshold (0.9).

Cross-implementation

The canonical alignment_verdict.json vector is regenerated from the fixed engine (case 2 "payload of one step changed" → high/0.9; alignment states unchanged). This is the canonical source the Swift port vendors and its vector-sync check enforces, so Swift #71 depends on this landing on main.

Tests

  • New tests/test_regression_risk_soundness.py (5): materially-changed critical, skipped-critical-masked-by-decoy, reordered criticals, benign structural reorder (no false HIGH), and the equivalent-step control.
  • test_regression_gate.py: the lenient-mode test is corrected — allow_divergent_steps=True relaxes the per-step check but not the severity gate, so a flipped critical decision now (correctly) fails unless max_regression_level is raised. Split off a genuinely-benign non-critical case to keep the "lenient tolerates benign changes" coverage.

Full suite: 480 passed, 3 skipped (venv). Behavior change: runs that previously (incorrectly) reported NONE may now report HIGH.

…ness)

Parity port of the Swift fix. The risk verdict was derived only from
removed/reordered alignment states, but a critical step that was materially
changed or skipped binds to a same-type event (type match alone clears the
matcher threshold), is classified AMBIGUOUS, and so escaped the verdict — a
tampered or skipped critical decision reported RegressionLevel.NONE on the
strict_audit_v1 profile, the exact regressions the engine exists to catch.

The verdict now derives from the equivalence outcome: a critical step that is
removed (0.95), reordered relative to another CRITICAL step (1.0), or bound
with a differing payload scoring below the profile's semantic_threshold
("changed beyond equivalence", 0.9) fires HIGH. Reorder is computed over
critical pairs only, so a benign structural/diagnostic step moving past a
stationary critical no longer fires a false HIGH.

The canonical alignment_verdict vector is regenerated (case 2 -> high/0.9;
states unchanged). test_regression_risk_soundness.py pins the four defects
plus the equivalent-step control. The RegressionGate lenient-mode test is
updated: allow_divergent_steps relaxes the per-step check but not the
severity gate, so a flipped critical decision now (correctly) fails unless
max_regression_level is raised.

Behavior change: runs that previously (incorrectly) reported NONE may now
report HIGH.
@Therealdk8890
Therealdk8890 changed the base branch from fix/deep-review-confirmed-findings to main July 18, 2026 00:51
@Therealdk8890
Therealdk8890 merged commit b8ed9b0 into main Jul 18, 2026
1 check passed
Therealdk8890 added a commit that referenced this pull request Jul 18, 2026
The RegressionRisk soundness fix (#53) replaced the AlignmentStateKind.REORDERED
lookup with a critical-pairs computation, orphaning the import. Ruff F401.
@Therealdk8890
Therealdk8890 deleted the fix/regression-risk-soundness branch July 18, 2026 06:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant