Repository navigation
refactor(test-writing): correct unapplicable rules and scope fix remediation - #180
Merged
Martin Bens (SpiGAndromeda) merged 3 commits intoSep 6, 2026
Merged
Conversation
Several rules in the test catalog generated findings that fail when applied: remediations the target repository's static gates reject, intentional patterns the rules could not see, and re-derivations of checks that active PHPStan rules already make deterministically. UNIT-004 gains an exception for a shared `createMock()` double whose sibling methods set expectations, and a deferral (echoed in UNIT-003) to `NoCreateMockWithoutExpectationsRule` where that rule already reports a double. Both are tied to the `createMockWithoutExpectationsEnabledNamespaces` regime in Shopware's PHPStan config. A `never()`/`once()` pairing across two test methods is recognized as the assertion that pins branch reachability, exempt from the over-coupling remedy. Remediation notes quote the Shopware unit-test guideline's two stub-conversion gotchas (`->with()` exists on mocks only, and a fixture-helper parameter takes `Foo&Stub`, not `MockObject`), because a remedy omitting them passes PHPUnit and fails PHPStan. CONV-007 and MIGRATION-007 are removed: php-cs-fixer's `ordered_class_elements` runs over `tests/` and enforces CONV-007's member slots mechanically, and phpstan-phpunit's `AssertEqualsIsDiscouragedRule` is active on every analyzed path including `tests/migration`. MIGRATION-008's comment no longer references the removed rule. CONV-005 drops to `consider` and gains a single-category exemption, so a class deliberately holding one half of a split is no longer told to grow the other half. CONV-014 treats an assertion between two actions as intentional when moving it past the next action changes the observed value, and its own Fix example no longer performs that defective move. DESIGN-003 compares assertions rather than bodies and requires a provider fold to respect the target provider's documented scope. DESIGN-004 keeps boundary inputs whose distinctness the implementation deliberately erases. CONV-011 suppresses annotations that mechanically expand the method name. UNIT-001 findings name the mechanism enforcing an assertion called trivial. CONV-017 sends reused state to `setUp()` instead of inlining it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The reviewing skills, team-review prompts, and docs now match the reduced catalog. Composed class-structure claims name only CONV-005, the migration skill's should-fix example cites CONV-014 because CONV-005 is a consider rule now, the red-team lens shorthand drops UNIT-005 (removed in 4.2.1) and cites CONV-012 where it named MIGRATION-007, the README and AGENTS.md tables drop the removed rows, and the root README's count names what it counts ("the 43 rules of the Shopware unit-test catalog"). Assertions in `review_unit.bats` and `build_rule_package.bats` retarget their CONV-007 checks onto CONV-005 and CONV-017, preserving what each test pins (class-structure filtering, `scoped-review=exclude` handling, exact-id matching).
A remediation invariant lands in the three reviewing skills' output requirements and in the four team-review prompt builders (reviewer, reconcile, red team, defense): a suggested fix never changes what an existing assertion pins, and never introduces an assertion as a means of satisfying a structural, layout, naming, or style constraint. Coverage-gap rules (MIGRATION-008, DESIGN-006, INTEGRATION-007) keep new assertions as the finding's explicit deliverable, deletions ride `removed_assertions`, and re-expressing the same pinned fact (CONV-012's `assertTrue($a === $b)` to `assertSame($b, $a)`) stays legal. The migration skill's warnings example now carries an empty `deleted_methods`/`removed_assertions` payload, matching CONV-014's reorder-only semantics.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Agent Skills ExportBuilt 7 skill packages: |
Martin Bens (SpiGAndromeda)
deleted the
refactor/test-rule-catalog-refinement
branch
October 11, 2026 21:18
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rule-catalog corrections, two removals, and a remediation invariant for
test-writing, released as 5.2.0. The rule corrections implement the catalog items from #174; the CONV-005 ordering item is withdrawn there with a warning note. The remediation invariant relates to #176, whose structure-churn item stays open.Rules corrected against what they actually enforce
Each of these rules flagged code the project's own configuration permits, demanded something its cited mechanism does not check, or contradicted its own examples. Each fix quotes the deciding source.
shopware.createMockWithoutExpectationsactually enables it (Shopware\Tests\Unit\,Shopware\Tests\DevOps\, percommon.neon:73), and the mock-without-expectations mechanics defer to PHPStan'sNoCreateMockWithoutExpectationsRule. Anever()/once()pair that pins reachability is carved out with a worked example — the issue text is explicit that no outcome assertion substitutes for it. A "Stub Conversion Gotchas" section quotes the guideline verbatim (->with()exists on mocks only; the replacement type isFoo&Stub, notMockObject), because remediations kept producing stubs that no longer compile.considerand given a single-category exemption. A class whose methods all belong to one category has no ordering to violate.Removed: checks a deterministic gate already runs
CONV-007— class-element orderingordered_class_elementsovertests/MIGRATION-007—assertEqualsdiscouragedAssertEqualsIsDiscouragedRule, all pathsThe unit catalog goes from 44 rules to 43.
get_rules(ids=...)returns not-found for either id. Fallout is propagated: the composed class-structure claims name only CONV-005, the migration skill's should-fix example cites CONV-014, the red-team lens shorthand drops UNIT-005 (removed in 4.2.1) and cites CONV-012 where it named MIGRATION-007, and the BATS assertions that pinned CONV-007 behavior retarget onto CONV-005 and CONV-017 while keeping what each test pins.Remediation invariant
A reviewer's suggested fix could satisfy a structural rule by weakening what the test proves — reorder methods and drop an assertion, rename and retarget one, silence a layout finding by asserting something adjacent. The three reviewing skills' output requirements and the four team-review prompt builders (reviewer, reconcile, red team, defense) now carry the same invariant: a fix never changes what an existing assertion pins, and never introduces an assertion as a means of satisfying a structural, layout, naming, or style constraint. Coverage-gap rules (
MIGRATION-008,DESIGN-006,INTEGRATION-007) keep new assertions as the finding's explicit deliverable, deletions rideremoved_assertions, and re-expressing the same pinned fact stays legal (CONV-012'sassertTrue($a === $b)toassertSame($b, $a)). The migration skill's warnings example now carries emptydeleted_methods/removed_assertions, matching CONV-014's reorder-only semantics.Limits
tests/unitmethod names, 61% fell into the default category, so deterministic category identification is not possible and must not be reattempted. Test-rules catalog rules produce findings that cannot be applied as written #174 documents this under the CONV-005 bullet.