Repository navigation
test: stress, fuzz, and run the spawn where bounded tests stop - #39
Conversation
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueNote Reviews pausedUse the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
📜 Recent review details
📝 Walkthrough
Merge Risk: ⚪ Minimal · up to
|
| Check name | Status | Explanation |
|---|---|---|
| Title check |  |
The title uses the required Conventional Commits format and accurately describes the added stress, fuzz, and extended spawn testing. The wording is somewhat informal but remains specific and related t… |
| Description check | |
The description includes Summary, Validation, and Checklist sections. It documents the main changes, exact validation commands, Windows versions, test results, documentation updates, and completed che… |
| Linked Issues check | |
Direct issue #38 has coding requirements for expanded validation. The PR adds the ignored handle-count stress test with configurable cases and the required spawn modes. It adds Windows cargo careful… |
| Out of Scope Changes check | |
The changes stay within issue #38. Workflow changes implement the requested validation jobs and extended runs. Fuzz targets, domain cfg changes, stress-test support, random scaling, documentation, a… |
| Behavior Verification Evidence | |
The PR provides relevant verification evidence. The diff adds a 13-mode handle-count stress test, parser-model fuzz targets, scaled random tests, ARM64, cargo-careful, Miri, and AddressSanitizer CI jo… |
| Public Interface Compatibility | |
No supported breaking interface change is introduced. The library diff only adds documentation to the existing Command::env_clear method; it does not change public signatures or implementation. The … |
✨ Finishing Touches 💡 1
-
🛠️ Fix failing CI checks 💡
-
- Commit to this branch
- Create a new PR
🧪 Generate unit tests (beta)
-
- Commit to this branch
- Create a new PR
✨ Simplify code
-
- Commit to this branch
- Create a new PR
-
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Comment @coderabbitai help to get the list of available commands.
21416dc to
8fcc3dc
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
8fcc3dc to
72fc873
Compare
72fc873 to
92e11fe
Compare
|
@coderabbitai pause |
✅ Action performedReviews paused. |
The ignored test `spawn_cleanup_stress` spawns children in thirteen modes, `WINDOWS_SPAWN_STRESS_CASES` times in all. The modes cover both commits, a dropped suspended child, collected output, written input, a killed tree, transferred handles, a Job, an alternate parent, and two failed spawns. It requires the process's handle count to end where it started, after a warm-up that runs each mode twice. On Windows 11, 1300 spawns left the count at 86, where it started. `WINDOWS_SPAWN_RANDOM_SCALE` multiplies the random command lines and environments, which keep their seeds. `tests/argv_roundtrip.rs` propagates its errors instead of calling `expect`, and adopts handles without `as`. The file-wide lint exception and its three registry entries go away. Refs #38
A `fuzz` crate compiles `command_line_rules.rs` and `environment_rules.rs` from `src/domain`. Its `command-line` target requires both C runtime parser models to read back the program name and every argument it encodes. Its `environment-block` target requires a built block to end where `scan` finds its end and to read back as its variables, in order. The parser models and the block scan compile under `cfg(fuzzing)` for them, and the models name the backslash pair they count. The fuzz crate forms its own workspace, so the package, the MSRV, and the workspace lints leave it out. The repository gate checks its targets, and `REUSE.toml` states the license of its files. A broken encoder, which drops the backslashes before a closing quote, failed the `command-line` target within seconds. Refs #38
Every PR now runs 1300 stress spawns on x64 and i686, the Windows tests under `cargo careful`, and `cargo xtask test` on Windows 11 ARM64. It also runs Miri and AddressSanitizer over the platform-neutral decisions, and each fuzz target for ten minutes. `required` waits for each of these jobs. A nightly workflow runs the stress test 100000 times on x64 and 20000 times on i686. It also runs the random cases at fifty times their size, and each fuzz target for five hours. ADR 0017 records what each check covers and credits #14, and CONTRIBUTING shows how to run them. Refs #38
…cess The first ARM64 run showed that Windows on ARM64 gives every child `PROCESSOR_ARCHITECTURE`, even a child whose environment the command cleared. The random environment test now leaves that variable out, as it leaves out the variables that `appverif.exe` and the profile runtime set. `Command::env_clear` states the behavior. Refs #38
zizmor reported the matrix values that the stress and fuzz steps expanded into their scripts. The stress jobs now name their target through `CARGO_BUILD_TARGET`, and the fuzz jobs through `FUZZ_TARGET`, so no script expands a template. The nightly workflow also runs one deep run at a time. Refs #38
92e11fe to
b041a97
Compare
1 participant
Summary
Closes #38.
This PR stacks on #36.
It brings back the checks of #14 that main dropped, and each one now checks a property rather than an absence of panics.
spawn_cleanup_stressspawns children in thirteen modes and requires the process's handle count to end where it started.The modes cover both commits, a dropped suspended child, collected output, written input, a killed tree, transferred handles, a Job, an alternate parent, and two failed spawns.
cargo careful, whose standard library checks the preconditions of the unsafe operations insys.cargo xtask teston a Windows 11 ARM64 runner.fuzzcrate compiles the encoder and the block builder fromsrc/domain.Its targets require the parser models to read back every program name, argument, and variable they encode.
WINDOWS_SPAWN_RANDOM_SCALEmultiplies the random command lines and environments, which keep their seeds.tests/argv_roundtrip.rspropagates its errors instead of callingexpectand adopts handles withoutas.Its file-wide lint exception and three registry entries go away.
The first ARM64 run found that Windows on ARM64 gives every child
PROCESSOR_ARCHITECTURE, even a child whose environment the command cleared.The random environment test now leaves that variable out, and
Command::env_clearstates the behavior.Validation
On Windows 11, x64:
cargo xtask testpassed.cargo careful testpassed 70 unit tests, 34 integration tests, and 36 xtask tests.On macOS, Apple silicon:
command-lineand 8.9 million forenvironment-block.command-linetarget failed within seconds.cargo xtask gates, and the xtask tests passed.actionlint,reuse lint, andtypospassed.zizmorreports nothing on the new jobs.In this PR's CI:
cargo xtask testpassed 73 unit tests, 36 integration tests, 36 xtask tests, every Lean row, and every random case.command-lineand 32.2 million forenvironment-block.cargo carefulpassed.The nightly workflow runs once this PR reaches
main, because GitHub dispatches only a workflow that the default branch holds.The owner has paused CodeRabbit, so its review of this PR waits for the owner to resume it.
Checklist
@coderabbitai ignore