A failed assertion is never the expected throw (0.9.2)

CHANGELOG.md

@@ -1,5 +1,16 @@
 # Changelog
 
+## 0.9.2 — 2026-09-19
+
+- **Fixed.** A wide `throws()` swallowed a failed assertion: PHPUnit's
+  `AssertionFailedError` extends `\RuntimeException`, so
+  `->throws(\RuntimeException::class)` — or `\Exception`, `\Throwable` —
+  matched a failing `assertSame()` in the body and the property passed on a
+  falsified body, the T-1 of 0.9.0 on the new knob. `throws()` refuses a
+  class the assertion failure is an instance of, naming why; the executor
+  never takes an assertion failure for the expected throw, whatever class
+  was named (#54).
+
 ## 0.9.1 — 2026-09-19
 
 - **Changed.** Accepts `rasuvaeff/property-testing-core` `^0.11` alongside `^0.9`/`^0.10`. Nothing in 0.11.0 reaches this package: the release is additive (the `PropertyTestingException` marker, `AssumptionSkipped` promoted to `@api`, `Gen::stringOf()` defaults, docblock reading in `Gen::forClass()`), and the suite is green against it unchanged.

README.md

@@ -230,6 +230,12 @@ Semantics, per trial:
   but it was not`, and the input shrinks like any other counterexample.
 - Throws another class — that throw is the failure, exactly as it would be
   without `throws()`.
+- A failed assertion is never the expected throw. PHPUnit's
+  `AssertionFailedError` extends `\RuntimeException`, so `throws()` refuses
+  a class it is an instance of — `\RuntimeException`, `\Exception`,
+  `\Throwable` — with `a failed assertion is an instance of it`; name the
+  exception the body throws. A failing `assertSame()` inside the body fails
+  the trial whatever class was expected.
 - `markTestSkipped()`/`markTestIncomplete()` still skip the run, and an
   `Assume::that()` discard still discards it: the environment's verdict about
   the run is never a pass earned by throwing.

README.ru.md

@@ -228,6 +228,12 @@ $this->forAll(['width' => Gen::intBetween(1, 5000), 'minWidth' => Gen::intBetwee
 - Возвращается нормально — trial **валится** с `Expected <class> to be thrown,
   but it was not`, и вход shrink-ается как любой контрпример.
 - Бросает другой класс — этот бросок и есть падение, ровно как без `throws()`.
+- Упавшая ассерция — никогда не ожидаемый бросок. `AssertionFailedError`
+  PHPUnit наследует `\RuntimeException`, поэтому `throws()` отвергает класс,
+  экземпляром которого она является — `\RuntimeException`, `\Exception`,
+  `\Throwable` — с `a failed assertion is an instance of it`; называйте
+  исключение, которое бросает тело. Упавший `assertSame()` в теле валит
+  trial, какой бы класс ни ожидался.
 - `markTestSkipped()`/`markTestIncomplete()` всё равно пропускают прогон, а
   `Assume::that()` — discard'ит его: вердикт окружения о прогоне никогда не
   становится пасом, заработанным броском.

llms.txt

@@ -91,7 +91,7 @@ public function budgetMs(int $budgetMs): self;       // whole-phase wall-clock b
 public function examples(array $examples): self;     // list<list<mixed>> positional tuples, run first, never shrunk
 public function listeners(PropertyListener ...$listeners): self; // engine event observers
 public function auto(bool $auto = true): self;       // derive missing generators from the closure signature; forAll() map = partial overrides
-public function throws(string $exceptionClass): self; // class every trial must throw (subclass matches); no-throw fails with "Expected <class> to be thrown, but it was not"
+public function throws(string $exceptionClass): self; // class every trial must throw (subclass matches); no-throw fails with "Expected <class> to be thrown, but it was not"; a class AssertionFailedError is an instance of (\RuntimeException, \Exception, \Throwable) is refused — a failed assertion is never the expected throw
 public function output($stdout, $stderr): self;      // redirect reports/trace streams (for tests)
 public function check(\Closure $property): void;     // run the property
 ```

src/PhpUnit/PhpUnitTrialExecutor.php

@@ -4,6 +4,7 @@
 
 namespace Rasuvaeff\PropertyTesting\PhpUnit;
 
+use PHPUnit\Framework\AssertionFailedError;
 use PHPUnit\Framework\IncompleteTest;
 use PHPUnit\Framework\SkippedTest;
 use Rasuvaeff\PropertyTesting\AssumptionSkipped;
@@ -64,7 +65,10 @@ public function execute(array $arguments): TrialOutcome
             // was declared to produce.
             return TrialOutcome::skipped();
         } catch (\Throwable $failure) {
-            if ($this->expectedExceptionClass !== null && $failure instanceof $this->expectedExceptionClass) {
+            // A failed assertion is the run's failure whatever class was
+            // expected: `throws()` refuses a class it is an instance of, and
+            // this holds the line for a subclass the check cannot know (#54).
+            if ($this->expectedExceptionClass !== null && $failure instanceof $this->expectedExceptionClass && !$failure instanceof AssertionFailedError) {
                 return TrialOutcome::passed();
             }
 

src/PhpUnit/PropertyCheck.php

@@ -148,7 +148,9 @@
      *
      * This is the property-level replacement for `expectException()`, which
      * cannot work inside a `check()` closure: the executor observes every
-     * throw before PHPUnit's own expectation mechanism does. A skip
+     * throw before PHPUnit's own expectation mechanism does. A failed
+     * assertion is never the expected throw, and a class it is an instance
+     * of (`\Exception`, `\Throwable`) is refused here. A skip
      * (`markTestSkipped()`, `markTestIncomplete()`) and an
      * `Assume::that()` discard are still what they are — the environment's
      * verdict about the run, never a pass earned by throwing.
@@ -163,6 +165,15 @@
                 $exceptionClass,
             ));
         }
+        // PHPUnit's assertion failure is an `\Exception`: a class it is an
+        // instance of would match a failed `assertSame()` and pass a
+        // falsified body (#54).
+        if (is_a(AssertionFailedError::class, $exceptionClass, allow_string: true)) {
+            throw new \InvalidArgumentException(sprintf(
+                'Invalid expected exception class "%s": a failed assertion is an instance of it — a falsified body would pass; name the exception the body throws',
+                $exceptionClass,
+            ));
+        }
 
         $this->expectedExceptionClass = $exceptionClass;
 
@@ -357,7 +368,7 @@
      */
     public function listeners(PropertyListener ...$listeners): self
     {
         $this->listeners = array_values($listeners);
 
         return $this;
     }
@@ -529,7 +540,7 @@
         $generators = [];
 
         foreach ($this->generators as $name => $generator) {
             $name = (string) $name;
 
             if (!$generator instanceof ArbitraryInterface) {
                 throw new \InvalidArgumentException(sprintf(
@@ -583,7 +594,7 @@
         $classifications = $statistics->classifications;
         $checks = $statistics->checks;
 
         if ($classifications === [] || $checks <= 0) {
             return;
         }
 

tests/PropertyCheckTest.php

@@ -8,6 +8,7 @@
 use PHPUnit\Framework\Attributes\CoversClass;
 use PHPUnit\Framework\Attributes\CoversTrait;
 use PHPUnit\Framework\Attributes\DataProvider;
+use PHPUnit\Framework\ExpectationFailedException;
 use PHPUnit\Framework\IncompleteTestError;
 use PHPUnit\Framework\SkippedWithMessageException;
 use PHPUnit\Framework\TestCase;
@@ -174,9 +175,9 @@ public function testThrowsPassesWhenEveryTrialThrowsTheExpectedClass(): void
         $this->forAll(['value' => Gen::intBetween(0, 1_000)])
             ->runs(25)
             ->seed(9)
-            ->throws(\RuntimeException::class)
+            ->throws(\DomainException::class)
             ->check(static function (int $value): void {
-                throw new \RuntimeException('too big: ' . $value);
+                throw new \DomainException('too big: ' . $value);
             });
 
         self::assertGreaterThan($before, $this->numberOfAssertionsPerformed());
@@ -187,19 +188,62 @@ public function testThrowsAcceptsASubclassOfTheExpectedClass(): void
         $this->forAll(['value' => Gen::intBetween(0, 100)])
             ->runs(10)
             ->seed(3)
-            ->throws(\Exception::class)
+            ->throws(\LogicException::class)
             ->check(static function (int $value): void {
-                throw new \RuntimeException((string) $value);
+                throw new \DomainException((string) $value);
             });
     }
 
+    /**
+     * PHPUnit's assertion failure is an `\Exception`; a `throws()` naming a
+     * class it is an instance of would pass a falsified body (#54).
+     */
+    #[DataProvider('assertionBaseClassProvider')]
+    public function testThrowsRefusesAClassAFailedAssertionIsAnInstanceOf(string $class): void
+    {
+        $this->expectException(\InvalidArgumentException::class);
+        $this->expectExceptionMessage(sprintf('Invalid expected exception class "%s": a failed assertion is an instance of it', $class));
+
+        $this->forAll(['value' => Gen::intBetween(0, 10)])->throws($class);
+    }
+
+    /** @return iterable<string, array{string}> */
+    public static function assertionBaseClassProvider(): iterable
+    {
+        yield 'Exception' => [\Exception::class];
+        yield 'RuntimeException, the base of every PHPUnit exception' => [\RuntimeException::class];
+        yield 'Throwable' => [\Throwable::class];
+        yield 'AssertionFailedError' => [AssertionFailedError::class];
+    }
+
+    public function testAFailedAssertionIsNeverTheExpectedThrow(): void
+    {
+        // The executor's half of #54, through a subclass the check cannot
+        // refuse up front: ExpectationFailedException extends
+        // AssertionFailedError, and a body whose assertion fails must fail
+        // even when that very class is the one expected.
+        try {
+            $this->forAll(['value' => Gen::intBetween(0, 10)])
+                ->runs(5)
+                ->seed(3)
+                ->throws(ExpectationFailedException::class)
+                ->check(static function (int $value): void {
+                    self::assertSame($value, $value + 1);
+                });
+
+            self::fail('The property should have been falsified');
+        } catch (AssertionFailedError $failure) {
+            self::assertInstanceOf(PropertyViolationException::class, $failure->getPrevious());
+        }
+    }
+
     public function testThrowsFailsWhenTheBodyReturnsNormally(): void
     {
         try {
             $this->forAll(['value' => Gen::intBetween(0, 10)])
                 ->runs(10)
                 ->seed(3)
-                ->throws(\RuntimeException::class)
+                ->throws(\DomainException::class)
                 ->check(static function (int $value): void {});
 
             self::fail('The property should have been falsified');
@@ -209,7 +253,7 @@ public function testThrowsFailsWhenTheBodyReturnsNormally(): void
 
             $reason = $previous->getPrevious();
             self::assertInstanceOf(\RuntimeException::class, $reason);
-            self::assertSame('Expected RuntimeException to be thrown, but it was not', $reason->getMessage());
+            self::assertSame('Expected DomainException to be thrown, but it was not', $reason->getMessage());
         }
     }
 
@@ -219,9 +263,9 @@ public function testThrowsFailsWhenTheBodyThrowsAnotherClass(): void
             $this->forAll(['value' => Gen::intBetween(0, 10)])
                 ->runs(10)
                 ->seed(3)
-                ->throws(\RuntimeException::class)
+                ->throws(\DomainException::class)
                 ->check(static function (int $value): void {
-                    throw new \LogicException('wrong one');
+                    throw new \RangeException('wrong one');
                 });
 
             self::fail('The property should have been falsified');
@@ -231,7 +275,7 @@ public function testThrowsFailsWhenTheBodyThrowsAnotherClass(): void
 
             // The unexpected throw itself is the counterexample's failure,
             // exactly as it would be without throws().
-            self::assertInstanceOf(\LogicException::class, $previous->getPrevious());
+            self::assertInstanceOf(\RangeException::class, $previous->getPrevious());
         }
     }
 
@@ -244,10 +288,10 @@ public function testTheMissingThrowShrinksLikeAnyOtherCounterexample(): void
             $this->forAll(['value' => Gen::intBetween(0, 10_000)])
                 ->runs(100)
                 ->seed(42)
-                ->throws(\RuntimeException::class)
+                ->throws(\DomainException::class)
                 ->check(static function (int $value): void {
                     if ($value < 100) {
-                        throw new \RuntimeException('small enough');
+                        throw new \DomainException('small enough');
                     }
                 });
 
@@ -267,7 +311,7 @@ public function testASkipUnderThrowsIsStillASkipNotAPass(): void
 
         $this->forAll(['value' => Gen::intBetween(0, 10)])
             ->runs(5)
-            ->throws(\RuntimeException::class)
+            ->throws(\DomainException::class)
             ->check(static function (int $value): void {
                 self::markTestSkipped('no redis here');
             });