Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
140 changes: 139 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,130 @@ jobs:
cargo test --target i686-pc-windows-msvc --test windows_spawn
pseudoconsole_regular_stdio_stays_off_parent_pipes -- --exact

stress:
name: Spawn cleanup stress / ${{ matrix.target }}
strategy:
fail-fast: false
matrix:
target: [x86_64-pc-windows-msvc, i686-pc-windows-msvc]
runs-on: windows-latest
timeout-minutes: 30
env:
CARGO_BUILD_TARGET: ${{ matrix.target }}
WINDOWS_SPAWN_STRESS_CASES: "1300"
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: dtolnay/rust-toolchain@e2a55d2ffb04f378e9626c28d38b36d230d1e12f # v1
with:
toolchain: stable
targets: ${{ matrix.target }}
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
key: ${{ matrix.target }}
- run: cargo test --locked --test windows_spawn spawn_cleanup_stress -- --ignored --exact

careful:
name: Windows tests under cargo-careful
runs-on: windows-latest
timeout-minutes: 45
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: dtolnay/rust-toolchain@e2a55d2ffb04f378e9626c28d38b36d230d1e12f # v1
with:
toolchain: nightly-2026-10-13
components: rust-src
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
- uses: taiki-e/install-action@1168f6810a3cb30a3514bf5756a3b7f0422c6df8 # v2.87.28
with:
tool: cargo-careful@0.4.10
- run: >-
cargo +nightly-2026-10-13 careful test --workspace --all-targets --locked
-- --test-threads=1

arm64:
name: Windows 11 ARM64
runs-on: windows-11-arm
timeout-minutes: 30
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: dtolnay/rust-toolchain@e2a55d2ffb04f378e9626c28d38b36d230d1e12f # v1
with:
toolchain: stable
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
- run: cargo xtask test

miri:
name: Miri over the platform-neutral decisions
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: dtolnay/rust-toolchain@e2a55d2ffb04f378e9626c28d38b36d230d1e12f # v1
with:
toolchain: nightly-2026-10-13
components: miri, rust-src
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
- run: cargo +nightly-2026-10-13 miri test --package windows-spawn --lib --locked

sanitizer:
name: AddressSanitizer over the platform-neutral decisions
runs-on: ubuntu-latest
timeout-minutes: 30
env:
RUSTFLAGS: -Zsanitizer=address
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: dtolnay/rust-toolchain@e2a55d2ffb04f378e9626c28d38b36d230d1e12f # v1
with:
toolchain: nightly-2026-10-13
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
- run: >-
cargo +nightly-2026-10-13 test --package windows-spawn --lib --locked
--target x86_64-unknown-linux-gnu

fuzz:
name: Fuzz / ${{ matrix.target }}
strategy:
fail-fast: false
matrix:
target: [command-line, environment-block]
runs-on: ubuntu-latest
timeout-minutes: 30
env:
FUZZ_TARGET: ${{ matrix.target }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: dtolnay/rust-toolchain@e2a55d2ffb04f378e9626c28d38b36d230d1e12f # v1
with:
toolchain: nightly-2026-10-13
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
workspaces: fuzz
- uses: taiki-e/install-action@1168f6810a3cb30a3514bf5756a3b7f0422c6df8 # v2.87.28
with:
tool: cargo-fuzz@0.13.2
- run: >-
cargo +nightly-2026-10-13 fuzz run "$FUZZ_TARGET" --target x86_64-unknown-linux-gnu
-- -max_total_time=600 -max_len=16384
- if: failure()
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: fuzz-${{ matrix.target }}
path: fuzz/artifacts/${{ matrix.target }}
if-no-files-found: ignore

verifier:
name: Application Verifier
runs-on: windows-latest
Expand Down Expand Up @@ -192,7 +316,21 @@ jobs:
ci-required:
name: CI required
if: always()
needs: [hygiene, test, i686-smoke, verifier, linux, proofs, coverage, package]
needs:
- hygiene
- test
- i686-smoke
- stress
- careful
- arm64
- miri
- sanitizer
- fuzz
- verifier
- linux
- proofs
- coverage
- package
runs-on: ubuntu-latest
steps:
- if: contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled') || contains(needs.*.result, 'skipped')
Expand Down
93 changes: 93 additions & 0 deletions .github/workflows/deep-verification.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
name: Deep verification

"on":
schedule:
- cron: "17 3 * * *"
workflow_dispatch:

permissions:
contents: read

# One deep run at a time; a dispatch that meets a scheduled run waits for it.
concurrency:
group: ${{ github.workflow }}
cancel-in-progress: false

jobs:
stress:
name: Spawn cleanup stress / ${{ matrix.target }}
strategy:
fail-fast: false
matrix:
include:
- target: x86_64-pc-windows-msvc
cases: "100000"
- target: i686-pc-windows-msvc
cases: "20000"
runs-on: windows-latest
timeout-minutes: 180
env:
CARGO_BUILD_TARGET: ${{ matrix.target }}
WINDOWS_SPAWN_STRESS_CASES: ${{ matrix.cases }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: dtolnay/rust-toolchain@e2a55d2ffb04f378e9626c28d38b36d230d1e12f # v1
with:
toolchain: stable
targets: ${{ matrix.target }}
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
key: ${{ matrix.target }}
- run: cargo test --locked --test windows_spawn spawn_cleanup_stress -- --ignored --exact

random:
name: Random command lines and environments at fifty times their size
runs-on: windows-latest
timeout-minutes: 300
env:
RUSTFLAGS: --cfg windows_spawn_vectors
WINDOWS_SPAWN_RANDOM_SCALE: "50"
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: dtolnay/rust-toolchain@e2a55d2ffb04f378e9626c28d38b36d230d1e12f # v1
with:
toolchain: stable
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
- run: cargo test --locked --test argv_roundtrip --test command_line_parsers

fuzz:
name: Fuzz / ${{ matrix.target }}
strategy:
fail-fast: false
matrix:
target: [command-line, environment-block]
runs-on: ubuntu-latest
timeout-minutes: 330
env:
FUZZ_TARGET: ${{ matrix.target }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: dtolnay/rust-toolchain@e2a55d2ffb04f378e9626c28d38b36d230d1e12f # v1
with:
toolchain: nightly-2026-10-13
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
workspaces: fuzz
- uses: taiki-e/install-action@1168f6810a3cb30a3514bf5756a3b7f0422c6df8 # v2.87.28
with:
tool: cargo-fuzz@0.13.2
- run: >-
cargo +nightly-2026-10-13 fuzz run "$FUZZ_TARGET" --target x86_64-unknown-linux-gnu
-- -max_total_time=18000 -max_len=16384
- if: failure()
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: fuzz-${{ matrix.target }}
path: fuzz/artifacts/${{ matrix.target }}
if-no-files-found: ignore
16 changes: 16 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,22 @@ On Windows, `just verifier` runs every test under `appverif.exe`, which stops a
It needs an administrator and the `appverif.exe` of the Windows SDK, which the hosted Windows runners of CI also have.
Three tests misuse a handle on purpose, and each must stop, which shows the verifier watches the run.

Deeper checks need no change to the code, and ADR 0017 says what each covers:

```powershell
$env:WINDOWS_SPAWN_STRESS_CASES = "1300"
cargo test --test windows_spawn spawn_cleanup_stress -- --ignored --exact
cargo +nightly careful test --workspace --all-targets -- --test-threads=1
```

```sh
cargo +nightly miri test --package windows-spawn --lib
cargo +nightly fuzz run command-line -- -max_total_time=600 -max_len=16384
cargo +nightly fuzz run environment-block -- -max_total_time=600 -max_len=16384
```

`WINDOWS_SPAWN_RANDOM_SCALE` multiplies the number of random command lines and environments, which the nightly workflow raises to fifty times.

## Expectations

- Keep the documented ownership and cleanup behavior, including on errors.
Expand Down
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ unsafe_op_in_unsafe_fn = "deny"
unused_lifetimes = "deny"
unused_qualifications = "deny"
variant_size_differences = "deny"
unexpected_cfgs = { level = "deny", check-cfg = ["cfg(kani)", "cfg(windows_spawn_vectors)"] }
unexpected_cfgs = { level = "deny", check-cfg = ["cfg(fuzzing)", "cfg(kani)", "cfg(windows_spawn_vectors)"] }

[workspace.lints.clippy]
all = { level = "deny", priority = -1 }
Expand Down
3 changes: 2 additions & 1 deletion REUSE.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ path = [
"deny.toml",
"docs/**",
"examples/**",
"fuzz/**",
"justfile",
"proofs/**",
"public-api/**",
Expand All @@ -30,7 +31,7 @@ SPDX-FileCopyrightText = "2026 Yasunobu Sakashita"
SPDX-License-Identifier = "MIT OR Apache-2.0"

[[annotations]]
path = ["Cargo.lock", "proofs/quint/package-lock.json"]
path = ["Cargo.lock", "fuzz/Cargo.lock", "proofs/quint/package-lock.json"]
precedence = "override"
SPDX-FileCopyrightText = "NONE"
SPDX-License-Identifier = "CC0-1.0"
Expand Down
39 changes: 39 additions & 0 deletions docs/adr/0017-long-careful-and-fuzzed-runs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# 0017 — Long, careful, and fuzzed runs check what bounded tests can't

Status: accepted (2026-10-13)

## Context

The tests run each behavior a few times, in one configuration, on x64 and i686.
A handle that leaks once in many spawns, an unsafe precondition that holds by luck, a fault on ARM64, and an encoder fault beyond the vectors and the bounds of Kani all pass them.
PR #14 ran a stress test, `cargo careful`, an ARM64 smoke test, Miri, AddressSanitizer, and two fuzz targets, and main dropped them with #14.

## Decision

- An ignored test, `spawn_cleanup_stress`, spawns children in thirteen modes.
The modes cover both commits, a dropped suspended child, collected output, written input, a killed tree, transferred handles, a Job, an alternate parent, and two failed spawns.
It requires the process's handle count to end where it started, after a warm-up that runs each mode twice.
`WINDOWS_SPAWN_STRESS_CASES` sets the number of spawns: 1300 on every PR on x64 and i686, and 100000 on x64 and 20000 on i686 every night.
- CI runs the Windows tests under `cargo careful`, whose standard library checks the preconditions of the unsafe operations in `sys`, such as those of `slice::from_raw_parts`.
- CI runs `cargo xtask test` on a Windows 11 ARM64 runner.
- A `fuzz` crate compiles the encoder and the block builder from `src/domain`, and requires the parser models to read back what they wrote.
The parser models and the block scan compile under `cfg(fuzzing)` for it.
Each target runs for ten minutes on every PR, and for five hours every night.
The fuzz crate is its own workspace, outside the package, the MSRV, and the workspace lints, and the repository gate checks its targets.
- Miri and AddressSanitizer run the platform-neutral decisions on Linux.
Those decisions hold no unsafe code, so these runs guard against a change that brings some in, at the cost of minutes.
- `WINDOWS_SPAWN_RANDOM_SCALE` multiplies the random command lines and environments, which keep their seeds, and the nightly workflow runs them at fifty times their size.

Rejected:

- The fuzz targets of #14, which checked only that the code doesn't panic.
- A stress test that measures time, which ADR 0010 bans.
A handle count decides by state.
- ARM64 through emulation, which would test the emulator.

## Consequences

- CI gains six jobs, and `required` waits for each of them.
- A leak in every spawn fails a PR, and a rarer leak fails the nightly run, whose 100000 spawns let it add up.
- A failed fuzz run uploads the input that failed.
- The nightly workflow runs on its schedule or on demand, and it gates no PR.
4 changes: 4 additions & 0 deletions fuzz/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
/target
/corpus
/artifacts
/coverage
Loading
Loading