Skip to content

fix(bench): keep reports flowing without install check - #1534

Merged
zackees merged 2 commits into
mainfrom
fix/benchmark-publish-robustness
Sep 28, 2026
Merged

zackees merged 2 commits into
mainfrom
fix/benchmark-publish-robustness

Conversation

@zackees

@zackees zackees commented Sep 28, 2026

Copy link
Copy Markdown
Member

Summary

The benchmark was failing after completing measurements because its final fbuild install --check --json metadata probe is rejected for registry-pinned non-ESP32 platforms. That made a post-measurement metadata step block publication of otherwise valid timing data.

This reads package identities from the successful timed build output instead. The report now distinguishes matched, different, and unverified stacks. Missing or inconsistent identities across any cold trial are recorded as unverified and suppress only the comparison ratio; the cold/warm timing bars still publish. ESP32-S3 compares platform, framework, and toolchain; Uno compares framework and toolchain.

Validation

  • soldr cargo test -p fbuild-bench-fastled-examples --bin bench-build-comparison — 30 passed
  • soldr cargo clippy -p fbuild-bench-fastled-examples --bin bench-build-comparison -- -D warnings — passed
  • soldr cargo fmt --all -- --check — passed
  • Parser format checked against a successful benchmark log; offline metadata failure reproduced in failed run #36345385136

The updated benchmark workflow watches this source path, so merging this PR should automatically run and publish a fresh report.

@zackees
zackees merged commit 3d0682e into main Sep 28, 2026
19 checks passed
@fastled-project-sync fastled-project-sync Bot moved this to Triage in FastLED Tracker Sep 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Triage

Development

Successfully merging this pull request may close these issues.

1 participant