fix(devtools): run the corpus and the required check at the pool width - #4682
Conversation
Two workers made a full corpus run take about seven hours, and the runner's affected tier could not finish a stale-graph run inside its 3,600 s slot timeout, so no pull request could pass its required check. The width is one fixed constant sized to the pytest pool ceiling; the descriptor no longer overrides it, and the slot timeout is 7,200 s. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DNGJ3awfNrsLaMdHgQZvid
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Team Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5862467f1d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| [operations.verify_affected] | ||
| description = "Run Polylogue's affected-test verification plan" | ||
| exec = ["env", "POLYLOGUE_PYTEST_WORKERS=2", "devtools", "verify"] | ||
| exec = ["devtools", "verify"] |
There was a problem hiding this comment.
At this reviewed head, running devtools test tests/unit/devtools/test_verify.py::test_verify_quick_descriptor_accepts_the_declared_json_projection fails because line 305 still requires the old env POLYLOGUE_PYTEST_WORKERS=2 command. Update that contract expectation alongside this intentional descriptor change; otherwise any complete corpus run—and any affected run selecting this test—reports a verification failure.
AGENTS.md reference: AGENTS.md:L172-L173
Useful? React with 👍 / 👎.
…ist group suffix --dist=loadgroup reports path::test@group; pytest cannot collect that literal, so every rerun after a red corpus errored before running a test. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DNGJ3awfNrsLaMdHgQZvid
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 09782f89e6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| failed = [ | ||
| str(test["nodeid"]) | ||
| _report_nodeid_to_selector(str(test["nodeid"])) | ||
| for test in report.get("tests", []) | ||
| if isinstance(test, Mapping) and test.get("outcome") in {"failed", "error"} and test.get("nodeid") |
There was a problem hiding this comment.
When an xdist_group test fails initially and passes alone, this stores only its stripped selector in failed/flaky, but lines 396 and 409 later compare that selector with the original report node ID ending in @group. Consequently the verifier may exit green while the canonical report leaves the test outcome and failed count unchanged (and also increments the passed count). Preserve the original-to-selector mapping or normalize those later comparisons as well.
Useful? React with 👍 / 👎.
| "working_directory": cwd, | ||
| "environment": dict(env), | ||
| "timeout_seconds": 3600, | ||
| "timeout_seconds": 7200, |
There was a problem hiding this comment.
When the required check needs close to the new two-hour allowance, it can never receive that full allowance: .github/workflows/verify.yml still gives the entire job 120 minutes, including checkout, graph seeding, and the wait behind other tasks in the host's serialized pytest queue, while this task's own 7,200-second timer starts only after it reaches the front. GitHub can therefore kill a valid queued run before this deadline or its terminal receipt; either lower this inner timeout or raise the workflow timeout enough to cover queue wait and setup.
AGENTS.md reference: AGENTS.md:L151-L154
Useful? React with 👍 / 👎.
| if default_worker_args != ["--dist=loadgroup", "-n", str(CORPUS_MAX_WORKERS)]: | ||
| print(f"testmon-selection: managed verification does not default to {CORPUS_MAX_WORKERS} workers") |
There was a problem hiding this comment.
The anti-vacuity mutation named by this gate, setting CORPUS_MAX_WORKERS = 0, no longer makes it fail: _pytest_worker_args() and the expected value now both read the same constant, so they both produce -n 0 and execution continues. The gate therefore cannot detect the zero-worker default it explicitly claims to prevent; assert an independent nonzero or intended-width value here.
AGENTS.md reference: AGENTS.md:L192-L193
Useful? React with 👍 / 👎.
…checkout graph A test that spawns devtools test (the clock-guard collection tests) opened the outer run's testmon datafile with its own session and reset it: the 2026-09-05 corpus run ended with a one-test graph. A nested run now loads pytest with testmon disabled. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DNGJ3awfNrsLaMdHgQZvid
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3f1249b5d2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| #: than host cores or free RAM. Measured 2026-09-03 uncontended: 47 minutes for | ||
| #: 20,860 tests at eight workers; at two the same run takes about seven hours | ||
| #: and the required check cannot finish inside its slot timeout. | ||
| CORPUS_MAX_WORKERS = 8 |
There was a problem hiding this comment.
When the scheduled operations.verify_all route runs, .agentctl/project.toml still exports POLYLOGUE_PYTEST_WORKERS=2 and gives the operation only 14,400 seconds. _pytest_worker_args(maximum=CORPUS_MAX_WORKERS) preserves that lower explicit value, so this new eight-worker corpus default never applies to the nightly full run; given the change's documented seven-hour runtime at two workers, the operation will time out after four hours instead of publishing complete-corpus evidence.
Useful? React with 👍 / 👎.
| def _testmon_args(env: Mapping[str, str]) -> tuple[str, ...]: | ||
| """testmon flags for a focused run; none when nested in a managed run.""" | ||
| if env.get(HARNESS_RUN_ENV): | ||
| return ("-p", "no:testmon") |
There was a problem hiding this comment.
When a test starts a nested devtools test, this branch does not actually disable testmon: MANAGED_PLUGIN_ARGS has already loaded the entry point as pytest-testmon, while -p no:testmon blocks only the differently named testmon/pytest_testmon plugins. As python -m pytest --help specifies, -p name loads the given module or entry-point name and no: avoids loading that named plugin; using no:pytest-testmon (or omitting it from the managed plugin list) is required. Otherwise the nested run continues writing the checkout's shared testmon datafile and can again replace the outer corpus graph with its focused scope.
AGENTS.md reference: AGENTS.md:L163-L168
Useful? React with 👍 / 👎.
) ## Summary Fixes one cluster of the 2026-09-05 master corpus reds (106 failed / 20,455 passed): status and readiness readers, local-time tests, the twelve-verb command surface, and the descriptor contract. - fix(daemon): keep archive FTS readiness on bounded probes - fix(cli): keep status readable on a skewed tier and a live daemon - fix(devtools): keep the root command surface at twelve verbs - test: assert insight readiness by convergence and counts - test: pin the local timezone without an ambient TZDIR - test: prove the descriptor against the agentctl parser - test: budget the delete-authority client for a real 513-session route - test: compare a restored zone offset as a timedelta ## Verification The lane's focused runs on the 25 listed tests were green before the interruption; `devtools verify --quick` green before push. This PR was published by the coordinator from the lane's clean checkpoint after the agent hit its quota; the exact final focused result line was not captured. ## Residual risk Tests whose failure is caused by the corpus width change (#4682, descriptor argv) were updated to the merged behaviour. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01DNGJ3awfNrsLaMdHgQZvid --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
…rsor probe (#4695) ## Summary Four tests encoded behaviour changed by merged PRs: the queue runner's name in the slot refusal (#4684), the descriptor's `verify` exec without a width override (#4682), and the cursor-store stub's signature after the read-only spool probe (#4683). The descriptor's `verify_all` entry still pinned `POLYLOGUE_PYTEST_WORKERS=2`; it now follows the same constant as the affected tier. ## Verification `devtools test tests/unit/devtools/test_verify.py tests/unit/devtools/test_dev_loop_service.py tests/unit/devtools/test_pytest_slot.py tests/unit/daemon/test_daemon_cli.py::test_spool_pending_check_ignores_terminal_cursor_states` green (result line in the pre-push run). From the 2026-09-05 corpus on 754553b (85 failed / 20,522 passed). 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01DNGJ3awfNrsLaMdHgQZvid Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
1 participant
Summary
The corpus and the runner's affected tier run at one fixed width of eight workers, sized to the pytest pool's 12 GiB cgroup ceiling. The descriptor no longer overrides the width, the pytest slot launch timeout is 7,200 s, and the selection gate reads the width from the constant.
Problem
CORPUS_MAX_WORKERS = 2made a full corpus run take about seven hours (measured 2026-09-05: 5% after 40 minutes), and the runner's affected tier could not finish a stale-graph run inside its 3,600 s slot timeout, so every pull request's requiredverifycheck timed out (runs 3669329 and 600822, both at 38%).Solution
Width 8 (47 minutes for 20,860 tests, measured uncontended 2026-09-03), one constant for both tiers, timeout 7,200 s.
Verification
devtools verify --quick: all gates ok, includingtestmon-selectionreportingworkers=8.Residual risk
Bead polylogue-m3018 asked for a modest width after eight workers plus lane-side pytest runs drove oomd kills on 2026-09-04; lane-side pytest is now refused (#4540, #4677) and the pool ceiling bounds the run. Browser responsiveness during a corpus run is the receipt m3018 still wants.
Beads: polylogue-m3018
🤖 Generated with Claude Code
https://claude.ai/code/session_01DNGJ3awfNrsLaMdHgQZvid