From 2065c3410bb257d9cb092f2ae98060a1094abd34 Mon Sep 17 00:00:00 2001
From: Amin Chirazi <32016576+AminChirazi@users.noreply.github.com>
Date: Sun, 2 Aug 2026 00:08:19 +0400
Subject: [PATCH] fix(cli): a test that refused to pass vacuously failed
 instead
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

`a_run_that_was_not_contained_keeps_no_evidence_from_an_optimistic_probe`
asserts that a control record carries what the RUN achieved rather than what
the host probe optimistically reported. To stop that passing for the wrong
reason it required the probe and the run to disagree — by asserting the
probe says `Enforced`.

That is only true on Linux. Everywhere else the assertion fired and the test
failed outright, so `cargo test --workspace` was red before anyone had
touched anything. The instinct was right; the mechanism was wrong.

The disagreement is what the test needs, not which side of it this machine
is on. Off Linux the probe still says "not contained" — but for a different
reason — so comparing the two rendered lines keeps the test honest
everywhere and green everywhere it is honest.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---
 CHANGELOG.md                    | 23 +++++++++++++++++++++++
 crates/flowproof-cli/src/lib.rs | 21 ++++++++++++++-------
 2 files changed, 37 insertions(+), 7 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 2f4fdfc..3328254 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -6,6 +6,29 @@ together).
 
 ## Unreleased
 
+### Fixed
+
+- **A test that refused to pass vacuously failed instead, and it was
+  red-lighting every non-Linux machine.**
+  `a_run_that_was_not_contained_keeps_no_evidence_from_an_optimistic_probe`
+  asserts that a control record carries what the RUN achieved rather than
+  what the host probe optimistically reported. To stop that passing for the
+  wrong reason, it required the probe and the run to disagree - by asserting
+  the probe says `Enforced`.
+
+  That is only true on Linux. Everywhere else the assertion fired and the
+  test failed outright, so `cargo test --workspace` was red before anyone
+  had touched anything. The instinct was right and the mechanism was wrong:
+  a test that cannot be meaningful on this host should say so, not fail.
+
+  The disagreement is what the test needs, not which side of it the machine
+  is on. Off Linux the probe still says "not contained" - but for a
+  DIFFERENT reason - so comparing the two rendered lines keeps the test
+  honest everywhere and green everywhere it is honest.
+
+  A gate that is red by default is one people learn to read past, which is
+  the whole reason it matters that this one was.
+
 ### Added
 
 - **A contained run could delete your files and the report would not mention
diff --git a/crates/flowproof-cli/src/lib.rs b/crates/flowproof-cli/src/lib.rs
index fa7364e..0cbb28f 100644
--- a/crates/flowproof-cli/src/lib.rs
+++ b/crates/flowproof-cli/src/lib.rs
@@ -2026,15 +2026,22 @@ mod tests {
             "the fixture must actually carry a blocked lane, or this test proves nothing"
         );
 
-        // The probe on this host says `Enforced`; the RUN says it was not
-        // contained. They differ, so passing can only mean the run won.
-        assert!(
-            agent_flow::containment(&spec).is_enforced(),
-            "this test is only meaningful where the probe and the run DISAGREE; \
-             on a host whose probe already says 'not contained' it proves nothing"
-        );
         let achieved =
             flowproof_adapters::Containment::NotContained("the filters never installed".into());
+        // The probe and the RUN must DISAGREE, or passing proves nothing.
+        //
+        // This used to require the probe to say `Enforced`, which is true
+        // only on Linux - so the test failed outright on every other host
+        // and made `cargo test --workspace` red before anyone had touched
+        // anything. The disagreement is what matters, not which side of it
+        // this machine happens to be on: off Linux the probe still says
+        // "not contained", but for a DIFFERENT reason, and the record must
+        // carry the run's reason rather than the probe's.
+        assert_ne!(
+            agent_flow::containment(&spec).report_line(),
+            achieved.report_line(),
+            "the probe and the run must differ, or this test proves nothing"
+        );
         let record = build_control_record(
             &spec_path,
             &dir,