diff --git a/bench/fastled-examples/README.md b/bench/fastled-examples/README.md index 43de278aa..250f933c4 100644 --- a/bench/fastled-examples/README.md +++ b/bench/fastled-examples/README.md @@ -60,10 +60,13 @@ threshold: 50.00 ms`) and recorded under the top-level `max_warm_ms` key in the JSON report, with the breach list under `breaches`. The `fastled-examples` job in `.github/workflows/bench-205.yml` passes -`--max-warm-ms 50`. The 50 ms value gives roughly 25× headroom over -the current ~1-2 ms warm timings on developer hardware (and ~10 ms on -CI runners), comfortably absorbing runner noise without false -positives. The job runs on every PR whose changes touch +`--max-warm-ms 50`. Headroom over that gate is **host- and file-cache +dependent, not 25×**: measured warm timings are ~11 ms on the 2026-05 +Windows capture and 31-41 ms on a 16-core NixOS host (2026-11-01 +re-capture, see below), i.e. 1.2-4.5× headroom. Treat a warm number near +50 ms as a possible host-load or resolver regression, not as noise, and +compare against a same-day run instead of the archived tables. The job +runs on every PR whose changes touch `crates/fbuild-library-select/**`, `crates/fbuild-header-scan/**`, `bench/fastled-examples/**`, the shared `MiniFramework` fixture, or the workflow itself — gating those PRs on the resolver staying within @@ -83,11 +86,30 @@ release build: | BlurBenchmark | 827.46 | 10.48 | 79.0x | | Chromancer | 844.13 | 10.89 | 77.5x | -The warm path comfortably clears AC#5 (≤ +50 ms over current fbuild) at -~11 ms per example. The ~75x speedup reflects the cost asymmetry between +The warm path clears AC#5 (≤ 50 ms) at ~11 ms per example on that +Windows workstation. The ~75x speedup reflects the cost asymmetry between walking the FastLED `src/` tree (~1000 files) and a `FileKvStore` get + bincode decode of a serialized `Selection`. +Captured 2026-11-01 on NixOS / 16-core x86_64, `soldr cargo run +--release`, same `EXAMPLES` set, two consecutive runs at host load +2-15 (16 cores). The speedup column collapses because the cold path is +much faster here (~10x faster than the 2026-05 Windows numbers): + +| example | cold (ms) | warm (ms) | speedup | +|---------------|----------:|----------:|--------:| +| Blink | 80-94 | 32-39 | 2.1x | +| Pacifica | 76-79 | 33-34 | 2.2x | +| Animartrix | 91-91 | 36-41 | 2.2x | +| Audio | 78-79 | 32-33 | 2.4x | +| BlurBenchmark | 75-77 | 31-33 | 2.3x | +| Chromancer | 80-82 | 33-34 | 2.4x | + +Every row reported `hit=yes` and both runs exited 0 under +`--max-warm-ms 50`. Warm spread across the two runs (31.2-40.9 ms) tracks +host load, so a bare warm number is only meaningful beside the load it +was taken at. + ## Curated example set The harness intentionally runs a small representative subset rather than diff --git a/bench/fastled-examples/src/main.rs b/bench/fastled-examples/src/main.rs index 7d38835b9..33f296468 100644 --- a/bench/fastled-examples/src/main.rs +++ b/bench/fastled-examples/src/main.rs @@ -41,8 +41,9 @@ //! //! `--max-warm-ms ` enforces AC#5: each example whose warm timing //! exceeds the threshold causes the binary to exit 1 after the table is -//! printed. Wired in CI at 50 ms (`~25x` headroom over current ~1-2 ms -//! warm numbers — absorbs runner noise without false positives). +//! printed. Wired in CI at 50 ms; measured warm timings are ~11 ms on the +//! 2026-05 Windows capture and 31-41 ms on a 16-core Linux host (2026-11-01), +//! so the headroom is 1.2-4.5x, not 25x — see `README.md`. //! //! Refs: #205 Phase 7 (AC#5), #218.