Skip to content

fix(tests): repair the order- and load-dependent failures parallelism exposed - #4572

Merged
dieterolson merged 3 commits into
mainfrom
fix/parallel-test-isolation
Aug 19, 2026
Merged

fix(tests): repair the order- and load-dependent failures parallelism exposed#4572
dieterolson merged 3 commits into
mainfrom
fix/parallel-test-isolation

Conversation

@dieterolson

Copy link
Copy Markdown
Collaborator

Two failures that #4548 surfaced by running the suite under xdist. Neither is caused by the diffs that hit them — main is red on one of them right now.

1. E-stop state leak (real isolation bug)

main.control_context is a module-level singleton shared by the entire P1AM backend suite. test_estop_trigger cleared the latch as the last statement of its body:

response = client.post("/api/estop")
assert response.status_code == 200
assert "E-stop triggered" in response.json()["message"]
mock_estop.assert_called_once()
control_context.clear_estop()   # <- skipped if any assert above fails

Any assertion above it failing leaves E-stop engaged for whatever xdist schedules next on that worker. It surfaced as test_pid_tuning_tag_guards failing with assert "PID loop 1 CV tag 'TAG_255'" in 'E-stop active; output writes are inhibited.' — an unrelated assertion, in an unrelated file, only on some runs.

Reproduced deterministically before fixing rather than inferred: latch E-stop, then run the victim module → 1 failed; with the guard → 2 passed.

Fix is two-layer: the clear moves into a finally, and a new package conftest.py clears the latch on both boundaries of every test. Teardown alone would still leave each module's first test dependent on whatever ran before it — which is the coupling being removed. This is the same class as the import-time P1AM_DEV_NO_AUTH bug (#4061), whose fix was likewise a per-test fixture.

2. Wall-clock flake

test_100k_rows_remain_bounded_and_fast asserted elapsed < 0.5. Main tripped it at 0.5146 s under contention. The assertion guards against an accidental super-linear pass — which at 100k rows costs minutes, not tenths of a second — so any ceiling in this range catches the real regression equally well, while 0.5 s reports a scheduling accident as a code regression. Ceiling moves to 5.0 s with the reasoning inline; displayed_count == 2_000 remains the deterministic contract.

Verification

  • Full P1AM backend suite: 1252 passed, 6 skipped, both serial and under -n 4.
  • test_launch_monitor_linked_scatter.py: 9 passed.
  • ruff check + format clean on all three files.

🤖 Generated with Claude Code

@
fix(tests): repair the order- and load-dependent failures parallelism exposed

Two failures that #4548 surfaced by running the suite under xdist. Neither is
caused by the diffs that hit them.

E-stop state leak. `main.control_context` is a module-level singleton shared
by the whole P1AM backend suite, and `test_estop_trigger` cleared the latch as
the last statement of its body. Any assertion above it failing left E-stop
engaged for whatever xdist scheduled next on that worker, which reported as
test_pid_tuning_tag_guards failing with "E-stop active; output writes are
inhibited." instead of its own expected message. The clear moves into a
`finally`, and a new package conftest clears the latch on both boundaries of
every test so the coupling cannot come back - teardown alone would still leave
each module first test dependent on whatever ran before it.

Reproduced deterministically before fixing (latch E-stop, run the victim:
1 failed; with the guard: 2 passed). Full backend suite: 1252 passed, 6
skipped, both serial and under -n 4.

Wall-clock flake. test_100k_rows_remain_bounded_and_fast asserted
elapsed < 0.5, which main itself tripped at 0.5146 s under contention. The
assertion guards against an accidental super-linear pass, which at 100k rows
costs minutes rather than tenths of a second, so the ceiling moves to 5.0 s
with the reasoning inline. displayed_count == 2_000 remains the deterministic
contract.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@dieterolson
dieterolson enabled auto-merge (squash) August 19, 2026 01:31
codex-scheduled and others added 2 commits August 18, 2026 18:49
@
fix(ci): allowlist the fixture-only P1AM backend conftest

quality-gate Changed Test Assertion Check flags any changed test file with no
behavioral assertion and names the remedy: add a fixture-only pattern to
scripts/test_assertion_allowlist.txt. The new conftest holds one autouse
fixture and no test functions, so it has nothing to assert about; the
behaviour it protects is asserted by the modules it isolates. Entry carries
the rationale, matching the surrounding convention.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@
@
docs(spec): place the new changelog row in descending order

The §12 table is newest-first (1.17.46 above 1.17.45). The row was inserted
after its anchor rather than before it, so the column read 46, 48, 45. Moves
it above 1.17.46.

The remaining disorder on that line is the pre-existing pair of duplicate
1.17.40 rows glued together by a union merge, tracked as #4569; splitting them
needs a renumber decision and is deliberately not done here.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@
@dieterolson
dieterolson merged commit 86abc33 into main Aug 19, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant