Skip to content

ci: fractional board validation with exact-SHA full gate - #1457

Merged
zackees merged 7 commits into
mainfrom
feat/fractional-ci-fbuild
Sep 24, 2026
Merged

zackees merged 7 commits into
mainfrom
feat/fractional-ci-fbuild

Conversation

@zackees

@zackees zackees commented Sep 23, 2026

Copy link
Copy Markdown
Member

Scope

Migrates generated board CI to an ordinary ci-minimal Linux gate, literal ci-test (Linux plus Uno), and opt-in ci-full with 76 distinct board executions representing all 80 aliases plus Linux/Windows, Dylint, QEMU, acceptance, benchmark, and fmt/docs/MSRV/board/crate policy checks. The generator remains the source of truth, and full/release paths verify the requested exact SHA. Unlabeled board workflows are manual/reusable only, avoiding the previous per-board PR/main fan-out.

This is a draft under zackees/soldr#3345. Publication intentionally fails closed until a trusted physical-board runtime result exists; no tag or release should be created from this PR. Live ci-test/ci-full runs and matched PR/main runner-minute measurements are still required before merge. Independent policy checks remain separately required in branch protection alongside CI selected coverage.

Local evidence: 19 focused tests and 95 subtests passed; generated workflow drift check and whitespace check passed. Independent pre-push review found no high-severity correctness issue. The local .venv has an unrelated setuptools warning, so tests were run via uv run --no-project --with pytest --with pyyaml.

@coderabbitai

coderabbitai Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Too many files!

This PR contains 106 files, which is 6 over the limit of 100.

To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch.

Upgrade to a paid plan to raise the limit.

This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry.

⚙️ Run configuration

Configuration used: Repository: FastLED/fbuild/.coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 9fd7da6d-426b-4ccb-b503-8cbe5e11ddcc

📥 Commits

Reviewing files that changed from the base of the PR and between 057408f and b9592b6.

📒 Files selected for processing (106)
  • .github/workflows/README.md
  • .github/workflows/acceptance-205.yml
  • .github/workflows/bench-205.yml
  • .github/workflows/build-apollo3_red.yml
  • .github/workflows/build-apollo3_thing_explorable.yml
  • .github/workflows/build-atmega8.yml
  • .github/workflows/build-atmega8a.yml
  • .github/workflows/build-attiny1604.yml
  • .github/workflows/build-attiny1616.yml
  • .github/workflows/build-attiny4313.yml
  • .github/workflows/build-attiny85.yml
  • .github/workflows/build-attiny88.yml
  • .github/workflows/build-blackpill.yml
  • .github/workflows/build-bluepill.yml
  • .github/workflows/build-ch32l103.yml
  • .github/workflows/build-ch32v003.yml
  • .github/workflows/build-ch32v006.yml
  • .github/workflows/build-ch32v103.yml
  • .github/workflows/build-ch32v203.yml
  • .github/workflows/build-ch32v208.yml
  • .github/workflows/build-ch32v303.yml
  • .github/workflows/build-ch32v307.yml
  • .github/workflows/build-ch32x035.yml
  • .github/workflows/build-clearcore.yml
  • .github/workflows/build-due.yml
  • .github/workflows/build-esp32c2.yml
  • .github/workflows/build-esp32c3.yml
  • .github/workflows/build-esp32c5.yml
  • .github/workflows/build-esp32c6.yml
  • .github/workflows/build-esp32dev.yml
  • .github/workflows/build-esp32h2.yml
  • .github/workflows/build-esp32p4.yml
  • .github/workflows/build-esp32s2.yml
  • .github/workflows/build-esp32s3.yml
  • .github/workflows/build-esp8266.yml
  • .github/workflows/build-giga-r1.yml
  • .github/workflows/build-leonardo.yml
  • .github/workflows/build-lpc804.yml
  • .github/workflows/build-lpc845.yml
  • .github/workflows/build-lpc845brk.yml
  • .github/workflows/build-lpcxpresso804.yml
  • .github/workflows/build-lpcxpresso845max.yml
  • .github/workflows/build-matrix_portal_m4.yml
  • .github/workflows/build-mgm240.yml
  • .github/workflows/build-nano-every.yml
  • .github/workflows/build-nano_every.yml
  • .github/workflows/build-nice_nano_nrf52840.yml
  • .github/workflows/build-nrf52840-sense.yml
  • .github/workflows/build-nrf52840_dk.yml
  • .github/workflows/build-nrfmicro_nrf52840.yml
  • .github/workflows/build-nucleo-f429zi.yml
  • .github/workflows/build-nucleo-f439zi.yml
  • .github/workflows/build-nucleo_f429zi.yml
  • .github/workflows/build-nucleo_f439zi.yml
  • .github/workflows/build-qtpy_m0.yml
  • .github/workflows/build-rp2040.yml
  • .github/workflows/build-rp2350.yml
  • .github/workflows/build-rpipico.yml
  • .github/workflows/build-rpipico2.yml
  • .github/workflows/build-sam3x8e_due.yml
  • .github/workflows/build-samd21.yml
  • .github/workflows/build-samd21_zero.yml
  • .github/workflows/build-samd51j.yml
  • .github/workflows/build-samd51p.yml
  • .github/workflows/build-stm32f103c8.yml
  • .github/workflows/build-stm32f103cb.yml
  • .github/workflows/build-stm32f103tb.yml
  • .github/workflows/build-stm32f411ce.yml
  • .github/workflows/build-stm32h747xi.yml
  • .github/workflows/build-supermini_nrf52840.yml
  • .github/workflows/build-teensy30.yml
  • .github/workflows/build-teensy31.yml
  • .github/workflows/build-teensy32.yml
  • .github/workflows/build-teensy35.yml
  • .github/workflows/build-teensy36.yml
  • .github/workflows/build-teensy40.yml
  • .github/workflows/build-teensy41.yml
  • .github/workflows/build-teensylc.yml
  • .github/workflows/build-thingplusmatter.yml
  • .github/workflows/build-tinystm.yml
  • .github/workflows/build-uno-r4-wifi.yml
  • .github/workflows/build-uno.yml
  • .github/workflows/build-uno_r4_wifi.yml
  • .github/workflows/build.yml
  • .github/workflows/check-macos.yml
  • .github/workflows/check-ubuntu.yml
  • .github/workflows/check-windows.yml
  • .github/workflows/ci-full.yml
  • .github/workflows/ci-minimal.yml
  • .github/workflows/ci-test.yml
  • .github/workflows/crate-gate.yml
  • .github/workflows/docs.yml
  • .github/workflows/dylint.yml
  • .github/workflows/fmt.yml
  • .github/workflows/msrv.yml
  • .github/workflows/nightly-platforms.yml
  • .github/workflows/qemu-linux-runtime.yml
  • .github/workflows/release-auto.yml
  • .github/workflows/template_build.yml
  • .github/workflows/validate-boards.yml
  • CLAUDE.md
  • ci/board_families.json
  • ci/render_workflows.py
  • ci/test_fractional_workflows.py
  • crates/fbuild-python/tests/pyo3_policy.rs
  • docs/RELEASING.md

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@zackees zackees added ci-test Run extended CI tests beyond the routine minimal gate and removed ci-test Run extended CI tests beyond the routine minimal gate labels Sep 23, 2026
@zackees zackees added the ci-full Run the complete release-equivalent CI matrix on this PR SHA label Sep 23, 2026
@zackees

zackees commented Sep 23, 2026 •

Copy link
Copy Markdown
Member Author

Full CI proof: hosted Apple Silicon and Intel macOS jobs are running in https://github.com/FastLED/fbuild/actions/runs/35914930520. The CH32V203 and ATtiny85 board cells are red because setup-soldr post-job yank-audit timed out after their build steps succeeded; tracked as zackees/setup-soldr#517. Full coverage remains red until that blocker is corrected.

@zackees

zackees commented Sep 23, 2026

Copy link
Copy Markdown
Member Author

The current exact-SHA ci-full run 35914930520 has now passed both hosted native macOS Rust workspace jobs (macos-15 Apple Silicon and macos-15-intel). Full coverage is still red because CH32V203 and ATtiny85 board jobs failed only in setup-soldr@v0 post-job yank-audit, tracked by setup-soldr #517 and draft fix #518. Native Mac proof is green; board/release approval is not. No tag or publication.

@zackees zackees removed the ci-full Run the complete release-equivalent CI matrix on this PR SHA label Sep 23, 2026
@zackees

zackees commented Sep 23, 2026

Copy link
Copy Markdown
Member Author

Same-head-SHA routine-cost proof for cff5358: unlabeled run https://github.com/FastLED/fbuild/actions/runs/35920280818 passed with two executed jobs (Linux Check and CI selected coverage), totaling 22.93 runner-minutes. The ci-full run https://github.com/FastLED/fbuild/actions/runs/35914930520 executed 102 jobs totaling 598.82 runner-minutes, so routine use was 3.83% of that full run, comfortably below the 12.5% target. This is raw elapsed runner time, not a billing claim. Full correctness remains RED: CH32V203 and ATtiny85 board jobs failed in the old setup-soldr post-job yank audit, propagating to the full sentinel. setup-soldr #518 is now merged to main but its v0 tag remains unchanged; do not merge/release this PR based on the cost result alone.

@zackees zackees added the ci-full Run the complete release-equivalent CI matrix on this PR SHA label Sep 23, 2026
@zackees

zackees commented Sep 23, 2026

Copy link
Copy Markdown
Member Author

Downstream canary at eae134a: the shared full-board template alone now pins merged setup-soldr worker-lifecycle fix f63d8987580fd50667d95b19130e82d9691ddd75 (setup-soldr#517). The ordinary PR/main tier selection is unchanged. Local render-and-diff and all 11 fractional workflow tests pass. I applied ci-full and am checking the full matrix, especially CH32V203 and ATtiny85, before treating this as proven.

@zackees zackees added ci-test Run extended CI tests beyond the routine minimal gate and removed ci-full Run the complete release-equivalent CI matrix on this PR SHA ci-test Run extended CI tests beyond the routine minimal gate labels Sep 23, 2026
@zackees

zackees commented Sep 23, 2026

Copy link
Copy Markdown
Member Author

Evidence for PR head eae134a79400fad305c43679523063cacc41b6b0:

  • Full ci-full run 35925056684 passed. Both previously red boards, CH32V203 and ATtiny85, passed their Post Setup soldr steps with the shared board template pinned to merged setup-soldr commit f63d8987580fd50667d95b19130e82d9691ddd75. Full coverage and CI selected coverage passed. Native macos-15-intel and macos-15 test jobs passed in this full tier.
  • Same-SHA unlabeled ordinary run 35927634754 passed: Linux and CI selected coverage passed; test and full were skipped, so no hosted Mac ran.
  • Literal ci-test label run 35929720490 passed: Linux, Arduino Uno, ci-test coverage, and CI selected coverage passed; full was skipped. The label was removed afterward. The latest same-SHA unlabeled run 35930729472 is still running its Linux job at the time of this comment; its final coverage result remains to be checked.

Raw runner usage from GitHub job start/end timestamps, summing non-skipped jobs: selected workflow ordinary 21.567 minutes versus full 736.433 minutes. The ten independent PR workflows triggered by the same head update added 16.483 minutes to either tier. Thus the all-workflow comparison is 38.050 / 752.917 = 5.05%, below the 12.5% goal. The superseded canceled selected run 35925053966 is excluded from both sides. These are raw runner minutes, without billing multipliers, and describe PR events at this SHA; main-push cost remains a separate acceptance check.

Main branch protection now requires both existing Dylint and the stable CI selected coverage check (GitHub Actions app ID 15368, strict mode). That sentinel runs for every PR tier and fails if the selected tier fails or, for ci-full, if full coverage is incomplete. A stale failed check visible in the PR rollup belongs to superseded canceled run 35925053966; the three completed tier runs above have green coverage checks. Because a later label removal can replace a red full result with a green minimal result on the same SHA, keep ci-full applied to platform-sensitive work until its failure is resolved.

This PR remains draft and unmerged. Current main (057408f3) is seven commits ahead of the common base; a read-only git merge-tree --write-tree HEAD origin/main reports a clean merge, but branch protection reports BEHIND and the branch needs an update plus fresh full proof before merge. Root CLAUDE.md already documents hosted Intel and Apple Silicon macOS for ci-full/release only; ordinary PR/main CI has no hosted Mac.

@zackees

zackees commented Sep 23, 2026

Copy link
Copy Markdown
Member Author

Integration correction: I restored main branch protection to its original Dylint-only required context (strict=true, GitHub Actions app 15368). Requiring CI selected coverage before this draft workflow reaches main would block unrelated PRs that cannot emit that new check. The coverage sentinel remains green in the tested PR runs and will be added as a required context immediately after this workflow is merged and main emits it. No CI evidence or branch SHA changed.

@zackees

zackees commented Sep 23, 2026

Copy link
Copy Markdown
Member Author

Final same-SHA label-removal proof: unlabeled run 35930729472 completed SUCCESS at eae134a79400fad305c43679523063cacc41b6b0; linux / Check (ubuntu-latest) and stable CI selected coverage both succeeded, and no ci-test/ci-full label remains. Full 35925056684, ordinary 35927634754, and literal ci-test 35929720490 also passed at this SHA. The draft is BEHIND current main under strict branch protection; update and fresh full proof are required before merge. Branch protection was restored to its prior Dylint-only context until this new coverage workflow reaches main, to avoid blocking unrelated PRs.

@zackees zackees added the ci-full Run the complete release-equivalent CI matrix on this PR SHA label Sep 24, 2026
@zackees

zackees commented Sep 24, 2026

Copy link
Copy Markdown
Member Author

setup-soldr floating-tag promotion completed safely:

Only the floating v0 tag moved. Immutable version tags were untouched. This delivers the fail-closed yank-audit/log-download retry fix to the fleet.

@zackees
zackees marked this pull request as ready for review September 24, 2026 02:32
@zackees
zackees merged commit 2a08abd into main Sep 24, 2026
115 checks passed
@zackees

zackees commented Sep 24, 2026

Copy link
Copy Markdown
Member Author

Merged-main measurement for 2a08abd4bc80d8803a9dcfd6c3fc1707b0fd2afb:

  • ordinary push: 36.10 runner-minutes across 10 executed jobs;
  • matched exact-head ci-full proof at b9592b6cf74732bb2659339fc27c15f6aa90edeb: 456.07 runner-minutes across 109 executed jobs;
  • ordinary/full ratio: 7.92%, below the 10% fleet target.

The ordinary ci-minimal Linux check and all lightweight policy workflows passed.

@fastled-project-sync fastled-project-sync Bot moved this to Triage in FastLED Tracker Sep 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-full Run the complete release-equivalent CI matrix on this PR SHA

Projects

Status: Triage

Development

Successfully merging this pull request may close these issues.

2 participants