Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,22 @@
# Changelog

## 0.10.0 — 2026-09-20

- **Added:** the core 0.12 knobs on the chain — `exhaustive()` /
`exhaustiveBudget()` (walk the parameter domain instead of sampling it),
`flakyReplays()` (re-execute the minimised counterexample; a passing replay
is reported as flaky) and `searchRuns()` (targeted search after the random
phase for a body that calls `Target::maximize()`/`minimize()`); each
validates at the call, naming the property, like the other setters.
- **Added:** `PROPERTY_EXHAUSTIVE` (a flag) and `PROPERTY_SEARCH_RUNS` (a
non-negative integer, `0` switches the search off) dial the suite, the way
`PROPERTY_DERANDOMIZE` and `PROPERTY_RUNS` do.
- **Added:** every `Classify::tabulate()` table with its pairwise
intersections, the exhaustive outcome (walked, on stdout; declined and why,
on stderr) and the search report are printed beside the distribution line;
`PROPERTY_VERBOSE` logs every `TargetImproved` event.
- Requires `rasuvaeff/property-testing-core` `^0.12`.

## 0.9.2 — 2026-09-19

- **Fixed.** A wide `throws()` swallowed a failed assertion: PHPUnit's
Expand Down
23 changes: 20 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ read — inside an ordinary PHPUnit `TestCase`.

- PHP 8.3+
- [`phpunit/phpunit`](https://packagist.org/packages/phpunit/phpunit) `^11.5 || ^12.0 || ^13.0`
- [`rasuvaeff/property-testing-core`](https://packagist.org/packages/rasuvaeff/property-testing-core) `^0.9 || ^0.10 || ^0.11`
- [`rasuvaeff/property-testing-core`](https://packagist.org/packages/rasuvaeff/property-testing-core) `^0.12`

PHPUnit 13 requires PHP 8.4.1 or newer. On PHP 8.3, Composer resolves a
compatible PHPUnit 11 or 12 release.
Expand Down Expand Up @@ -113,12 +113,17 @@ Reproduce the exact run by pinning the reported seed: `->seed(7382910)`.
| `edgeCases(EdgeCases)` | `None` turns off the numeric boundary bias — for a property the edges only cost runs |
| `auto(bool = true)` | Derives generators from the closure's signature for every parameter the `forAll()` map does not cover; the map becomes partial overrides. Off by default, and stays off |
| `throws(string)` | The exception class every trial must throw — a trial that throws it passes, one that does not (or throws another class) fails and shrinks. The property-level replacement for `expectException()`, which never sees a throw from inside the body |
| `exhaustive(bool = true)` | Walk the whole parameter domain instead of sampling it when every generator is `Enumerable` and the product fits the budget; otherwise the phase samples and a warning says why. `runs()` is ignored when it walks — see the [core README](https://github.com/rasuvaeff/property-testing-core#exhaustive-mode) |
| `exhaustiveBudget(int)` | The largest domain `exhaustive()` walks (default 10 000) |
| `flakyReplays(int)` | Re-executions of the minimised counterexample (default 2); one that passes marks it flaky, with a `Flaky:` line in the failure. `0` disables — see [flaky detection](https://github.com/rasuvaeff/property-testing-core#flaky-detection) |
| `searchRuns(int)` | Bodies the targeted search may execute after the random phase, for a body that calls `Target::maximize()`/`minimize()` (default 0 — no search) — see [targeted search](https://github.com/rasuvaeff/property-testing-core#targeted-search-target) |

Every setter validates its argument at the call, with the property's name in
the message: `runs(0)` throws
`Property "testSortIsIdempotent": runs must be greater than or equal to 1`
(the same for `maxShrinks`/`maxDiscards` below `0` and for
`timeoutMs`/`budgetMs`/`shrinkBudgetMs` below `1`). A `path()` without a
(the same for `maxShrinks`/`maxDiscards`/`flakyReplays`/`searchRuns` below
`0` and for `timeoutMs`/`budgetMs`/`shrinkBudgetMs`/`exhaustiveBudget` below
`1`). A `path()` without a
`seed()` — in either order — or a `PROPERTY_PATH` without a seed is refused by
`check()` the same way. The `forAll()` map is checked by `check()` too, before
the engine runs: a value that is not an `ArbitraryInterface` throws
Expand Down Expand Up @@ -291,6 +296,8 @@ Byte-for-byte parity with the Testo adapter — one contract across adapters:
| `PROPERTY_DERANDOMIZE` | Derives every unset seed from the property id, making a whole suite reproducible without editing it. Same switch words as `PROPERTY_VERBOSE`: `''` is unset, `0`/`false`/`off`/`no` are off, anything else is on |
| `PROPERTY_PATH` | A recorded shrink descent (`CounterExample::$path`) replayed instead of searched for. Needs the seed that produced it; an explicit `path()` wins. It describes one failure, so run it with `--filter` on that one test — every other property would report the path as stale |
| `PROPERTY_EDGE_CASES` | `mixin` or `none` (case-insensitive) — the numeric boundary bias for the whole suite, overriding `edgeCases()`. An unknown value throws |
| `PROPERTY_EXHAUSTIVE` | Turns exhaustive mode on for every property whose domain fits its budget, overriding `exhaustive()`; the same switch words as `PROPERTY_DERANDOMIZE` |
| `PROPERTY_SEARCH_RUNS` | Non-negative integer overriding `searchRuns()` for the whole suite — a bigger search budget on a nightly, or `0` to switch it off. A malformed value throws |

`PROPERTY_DB` takes either a directory or a Redis DSN:

Expand Down Expand Up @@ -335,6 +342,16 @@ Property "testSortKeepsEveryElement" distribution: long 39% (77/200), short 61%
A property that discards more than 90% of its attempts (via `Assume::that()`)
gets a warning suggesting narrower generators.

Beside the distribution line, the adapter prints what else the engine
measured: each `Classify::tabulate()` table with its tag shares and the pairs
hit together (`Property "…" table features: compressed 40% (80/200), retried
15% (30/200); together: compressed & retried 10% (20/200)`), whether
exhaustive mode walked the domain (`enumerated its whole domain of 24
input(s)`, stdout) or why it sampled instead (stderr), and the search report
of a body that targets something (`search: 100 evaluation(s); delay max 58210
(7 improvement(s))`). `PROPERTY_VERBOSE` also logs every `TargetImproved`
event with the input that scored it.

These diagnostics — the distribution on stdout, the discard warning and the
unstable-id warnings on stderr — are written straight to the process streams
while PHPUnit is printing its progress dots, so each one starts with a newline
Expand Down
22 changes: 19 additions & 3 deletions README.ru.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ shrink до минимального контрпримера, который р

- PHP 8.3+
- [`phpunit/phpunit`](https://packagist.org/packages/phpunit/phpunit) `^11.5 || ^12.0 || ^13.0`
- [`rasuvaeff/property-testing-core`](https://packagist.org/packages/rasuvaeff/property-testing-core) `^0.9 || ^0.10 || ^0.11`
- [`rasuvaeff/property-testing-core`](https://packagist.org/packages/rasuvaeff/property-testing-core) `^0.12`

PHPUnit 13 требует PHP 8.4.1 или новее. На PHP 8.3 Composer выбирает
совместимый релиз PHPUnit 11 или 12.
Expand Down Expand Up @@ -113,12 +113,16 @@ Property falsified after 12 successful run(s); seed=7382910
| `edgeCases(EdgeCases)` | `None` выключает граничное смещение числовых генераторов — для property, которой края стоят только прогонов |
| `auto(bool = true)` | Достраивает генераторы из сигнатуры кложуры для параметров, не покрытых картой `forAll()`; карта становится частичными overrides. По умолчанию выключен и дефолтом не станет |
| `throws(string)` | Класс исключения, которое обязан бросить каждый trial: бросивший его trial проходит, не бросивший (или бросивший другой класс) валит property и shrink-ается. Замена `expectException()` на уровне property — сам `expectException()` бросок из тела не видит |
| `exhaustive(bool = true)` | Обойти весь домен параметров вместо выборки, когда каждый генератор — `Enumerable`, а произведение умещается в бюджет; иначе фаза делает выборку, а предупреждение говорит почему. `runs()` при обходе игнорируется — см. [README core](https://github.com/rasuvaeff/property-testing-core#exhaustive-mode) |
| `exhaustiveBudget(int)` | Наибольший домен, который обходит `exhaustive()` (по умолчанию 10 000) |
| `flakyReplays(int)` | Повторные исполнения минимизированного контрпримера (по умолчанию 2); прошедший помечает его flaky, со строкой `Flaky:` в сообщении. `0` выключает — см. [детекцию flaky](https://github.com/rasuvaeff/property-testing-core#flaky-detection) |
| `searchRuns(int)` | Сколько тел может исполнить целевой поиск после random-фазы, если тело вызывает `Target::maximize()`/`minimize()` (по умолчанию 0 — без поиска) — см. [целевой поиск](https://github.com/rasuvaeff/property-testing-core#targeted-search-target) |

Каждый сеттер проверяет аргумент в момент вызова, с именем property в
сообщении: `runs(0)` бросает
`Property "testSortIsIdempotent": runs must be greater than or equal to 1`
(так же `maxShrinks`/`maxDiscards` ниже `0` и `timeoutMs`/`budgetMs`/
`shrinkBudgetMs` ниже `1`). `path()` без `seed()` — в любом порядке — или
(так же `maxShrinks`/`maxDiscards`/`flakyReplays`/`searchRuns` ниже `0` и
`timeoutMs`/`budgetMs`/`shrinkBudgetMs`/`exhaustiveBudget` ниже `1`). `path()` без `seed()` — в любом порядке — или
`PROPERTY_PATH` без seed так же отвергает `check()`. Карту `forAll()`
`check()` тоже проверяет до запуска движка: значение, не являющееся
`ArbitraryInterface`, даёт
Expand Down Expand Up @@ -289,6 +293,8 @@ $this->forAll(['width' => Gen::intBetween(1, 5000), 'minWidth' => Gen::intBetwee
| `PROPERTY_DERANDOMIZE` | Выводит каждый незаданный seed из id property: весь сьют становится воспроизводимым без правки кода. Те же слова-переключатели, что у `PROPERTY_VERBOSE`: `''` — не задано, `0`/`false`/`off`/`no` — выключено, всё остальное — включено |
| `PROPERTY_PATH` | Записанный спуск shrink (`CounterExample::$path`) воспроизводится вместо повторного поиска. Нужен seed того прогона; явный `path()` побеждает. Он описывает одно падение, поэтому запускайте с `--filter` на этот один тест — любое другое property сообщит, что путь устарел |
| `PROPERTY_EDGE_CASES` | `mixin` или `none` (регистр не важен) — граничное смещение для всего сьюта, перекрывает `edgeCases()`. Неизвестное значение — исключение |
| `PROPERTY_EXHAUSTIVE` | Включает исчерпывающий режим для каждой property, чей домен умещается в бюджет, перекрывая `exhaustive()`; те же слова-переключатели, что у `PROPERTY_DERANDOMIZE` |
| `PROPERTY_SEARCH_RUNS` | Неотрицательное целое, перекрывающее `searchRuns()` для всего сьюта — больший бюджет поиска на ночном прогоне или `0`, чтобы выключить его. Испорченное значение — исключение |

`PROPERTY_DB` принимает либо каталог, либо Redis-DSN:

Expand Down Expand Up @@ -332,6 +338,16 @@ Property "testSortKeepsEveryElement" distribution: long 39% (77/200), short 61%
Property, отбрасывающая больше 90% попыток (через `Assume::that()`), получает
предупреждение с советом сузить генераторы.

Рядом со строкой распределения адаптер печатает остальное, что измерил
движок: каждую таблицу `Classify::tabulate()` с долями тегов и парами,
встретившимися вместе (`Property "…" table features: compressed 40% (80/200),
retried 15% (30/200); together: compressed & retried 10% (20/200)`), обошёл ли
исчерпывающий режим домен (`enumerated its whole domain of 24 input(s)`,
stdout) или почему сделал выборку (stderr), и отчёт поиска property, которая
что-то целит (`search: 100 evaluation(s); delay max 58210 (7 improvement(s))`).
`PROPERTY_VERBOSE` дополнительно логирует каждое событие `TargetImproved` с
давшим его входом.

Эта диагностика — распределение в stdout, предупреждение о discard'ах и
предупреждения о нестабильном id в stderr — пишется прямо в потоки процесса,
пока PHPUnit печатает точки прогресса, поэтому каждая строка начинается с
Expand Down
2 changes: 1 addition & 1 deletion composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
"require": {
"php": "8.3 - 8.5",
"phpunit/phpunit": "^11.5 || ^12.0 || ^13.0",
"rasuvaeff/property-testing-core": "^0.9 || ^0.10 || ^0.11"
"rasuvaeff/property-testing-core": "^0.12"
},
"require-dev": {
"ergebnis/composer-normalize": "^2.51",
Expand Down
6 changes: 6 additions & 0 deletions llms.txt
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,10 @@ public function phases(array $phases): self; // list<Phase>: Examples, C
public function derandomize(bool $on = true): self; // unset seed derived from the property id
public function path(string $path): self; // replay a recorded shrink descent; needs its seed
public function edgeCases(EdgeCases $mode): self; // Mixin (default) | None = no numeric boundary bias
public function exhaustive(bool $exhaustive = true): self; // walk the whole parameter product when every generator is Enumerable and it fits the budget
public function exhaustiveBudget(int $budget): self; // >= 1; default 10_000
public function flakyReplays(int $replays): self; // >= 0; default 2; a passing replay of the minimised input = flaky (Flaky: line)
public function searchRuns(int $runs): self; // >= 0; default 0 = no targeted search after the random phase (Target::maximize()/minimize())
public function seed(int $seed): self; // pin the random phase; disables corpus replay
public function maxShrinks(int $maxShrinks): self; // cap accepted shrink steps; 0 = no shrinking
public function maxDiscards(int $maxDiscards): self; // discard budget; default runs * 10
Expand Down Expand Up @@ -143,6 +147,8 @@ final class SortPropertyTest extends TestCase
| `PROPERTY_DERANDOMIZE` | same switch words as `PROPERTY_VERBOSE` | Unset seeds derived from the property id; `seed()`/`PROPERTY_SEED` still win |
| `PROPERTY_PATH` | recorded `CounterExample::$path` | Replays that shrink descent; needs the seed of that run; explicit `path()` wins |
| `PROPERTY_EDGE_CASES` | `mixin` \| `none` (case-insensitive) | Numeric boundary bias; OVERRIDES `edgeCases()`; unknown value throws |
| `PROPERTY_EXHAUSTIVE` | flag (same words as `PROPERTY_DERANDOMIZE`) | Exhaustive mode for the suite; OVERRIDES `exhaustive()` |
| `PROPERTY_SEARCH_RUNS` | non-negative integer (`0` = off) | Search budget for the suite; OVERRIDES `searchRuns()`; malformed throws |

### Classification output

Expand Down
Loading
Loading