Make ci-job-clippy's board selection cover all archs, dynamically - #3
Make ci-job-clippy's board selection cover all archs, dynamically#3ppannuto-claude wants to merge 2 commits into
Conversation
2c0f755 to
5db2edb
Compare
1507e44 to
89df5c2
Compare
…et-spec qemu_i486_q35's target is a bare JSON filename, not a path, which cargo only resolves as a custom target spec (rather than requiring a builtin triple name) via the unstable json-target-spec mechanism. Previously this was passed as a `-Zjson-target-spec` CLI flag baked into the board's own Makefile (`CARGO = cargo -Zjson-target-spec`), the only board doing so. Set it instead via `[unstable] json-target-spec = true` in the board's own .cargo/config.toml -- the same persistent-config mechanism ../cargo/riscv_flags.toml already uses for the RISC-V boards' JSON specs (those are referenced by relative path instead, so didn't need it spelled out explicitly). That makes a plain `cargo clippy`/`cargo check`/`cargo build` all work with no special-casing in the board Makefile, so the now-redundant CARGO override is dropped. Also removed a stale `[env] RUST_TARGET_PATH` + TODO comment that predated the current include-based config layout. Verified: a full `make -C boards/qemu_i486_q35` build still produces a working kernel binary with the CARGO override removed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
89df5c2 to
942da49
Compare
942da49 to
c92381f
Compare
| for arch in $(./tools/build/list_archs.sh); do | ||
| # Not every arch/* crate has a board (e.g. cortex-m3, today) -- skip | ||
| # those rather than erroring, same as the old board-first version did. | ||
| for board in "${boards[@]}"; do | ||
| if board_depends_on_arch "$board" "$arch"; then | ||
| echo "$board" | ||
| break | ||
| fi | ||
| done |
There was a problem hiding this comment.
Is cortex-m3 the only unused arch in-tree right now? There are PRs pending that will use it, if so, we can drop this and we'll pend this PR until the PRs land that add a board that use cortex-m3
There was a problem hiding this comment.
Confirmed: cortex-m3 is the only arch crate with zero coverage, direct or transitive -- checked no board/chip depends on it and no other arch crate depends on it either (both a board and an arch crate must be absent for this to fire; cortex-v7m/cortex-m/cortex-m0/riscv all have zero direct board matches too, but each is a sibling dependency of some other arch crate a board does use, so they don't trip this). Didn't find an open PR or issue referencing it via gh pr/issue list --search, but that's not conclusive if it's not filed yet. Implemented the hard-fail: make ci-job-clippy now fails specifically and only on cortex-m3 (verified), so this'll go green again on its own once a board landing depends on it.
c92381f to
67863e7
Compare
The hand-picked board list (nrf52840dk, raspberry_pi_pico, hifive1, qemu_i486_q35) silently missed arch/* crates with no board maintaining the list ever noticed: cortexm7, cortexm33, and rv64i had no clippy coverage with a real target. tools/build/list_arch_boards.sh picks one board per arch/* crate, so the set self-updates as boards and archs are added or removed instead of drifting stale. It iterates tools/build/list_archs.sh's small, already-existing arch list on the outside and boards on the inside, stopping at the first (alphabetically, since boards are sorted first) match per arch -- rather than scanning every board regardless of whether a match was already found. Every chip that depends on an arch crate does so directly (one hop; verified against every chips/*/Cargo.toml), and a few board crates also depend on an arch crate directly alongside their chip (e.g. hail, the apollo3 boards) -- so for each candidate board, check both its own Cargo.toml and its chip's for an arch/ path dependency. An arch crate with no board using it directly is only an error if no *other* arch crate depends on it either (checked with one more grep over arch/*/Cargo.toml): e.g. cortex-v7m, cortex-m, cortex-m0, and riscv all have zero boards depending on them directly today, but each is a sibling dependency of some other arch crate that a board does use directly (cortex-m4f -> cortex-v7m -> cortex-m, rv32i -> riscv, etc.), so building that board compiles -- and so checks -- them too as a side effect, with no separate pick needed. cortex-m3 is currently the only arch crate genuinely unreachable either way; the script correctly hard-fails on it (verified: `make ci-job-clippy` now fails specifically and only on cortex-m3, everything else still resolves to the same 7 boards as before). This should self-resolve once a board using cortex-m3 lands. Results are collected into a variable and only printed after the full loop succeeds, rather than printed as each arch resolves: a failure partway through then leaves stdout completely empty instead of an incomplete board list, which matters because a `$(shell ...)`-based caller (as opposed to a direct invocation checked with `|| exit 1`) can only observe empty-vs-nonempty output, not the exit status. An earlier version used `cargo metadata` to compute each board's whole transitive dependency graph, which turned out to be solving a problem that doesn't exist here -- the codebase just doesn't have deep chip->chip->arch chains that would need it. A cold-context review caught a real bug in the regex: `path = "..."` requires exact spacing around `=`, but two chips (psoc62xa, rp2040) write `path="..."` with none, so their arch dependency silently didn't match. Harmless today only by coincidence -- both boards using those chips also happen to redeclare the same arch crate directly, so the board-level lookup covered for it -- but a future cleanup removing that "redundant" direct dependency would silently drop cortex-m0p from clippy coverage with nothing to flag it. Loosened the regex to tolerate any whitespace around `=`. Verified: `make prepush` passes except for ci-job-clippy, which fails specifically and only on the cortex-m3 gap described above; output for every other arch is identical to the earlier board-first version's. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
67863e7 to
b12efe4
Compare
ppannuto
left a comment
There was a problem hiding this comment.
Note that changes from the first commit in this series have been cherry-picked to master and should be dropped from this PR
| # Variants of another board (extra feature/policy configs, tutorial | ||
| # copies) are skipped, same as boards/README.md's own tooling, since | ||
| # they're not independent ports and would just be redundant picks. | ||
| # Sorted once so the loop below can stop at the first (alphabetical) match. |
There was a problem hiding this comment.
| # Variants of another board (extra feature/policy configs, tutorial | |
| # copies) are skipped, same as boards/README.md's own tooling, since | |
| # they're not independent ports and would just be redundant picks. | |
| # Sorted once so the loop below can stop at the first (alphabetical) match. | |
| # Just look at "real" boards. Alpha-sort for consistency in arbitrary choices. |
Pull Request Overview
ci-job-clippy's per-board list (nrf52840dk, raspberry_pi_pico, hifive1,qemu_i486_q35) was a hand-picked, hand-maintained set meant to exercise
clippy against each
arch/*crate's real target-specific code (workspaceclippy alone runs against the host target and never touches
cfg(target_arch = "...")code). It quietly missed several archs: nothingcovered
cortexm7,cortexm33, orrv64i.Replaces the static list with
tools/build/list_arch_boards.sh, whichpicks one deterministic (alphabetically-first) representative board per
arch/*crate actually in use. The set self-updates as boards/archs areadded, removed, or rewired, instead of silently drifting out of sync. New
board set:
apollo3/lora_things_plus,arty_e21,cy8cproto_62_4343_w,imxrt1050-evkb,lpc55s69-evk,qemu_i486_q35,qemu_rv64_virt.An earlier version of this PR computed the board set via
cargo metadata(each board's full transitive dependency graph). Turned out to be
solving a problem the codebase doesn't have: every chip that depends on
an arch crate does so directly (verified against every
chips/*/Cargo.toml),so the script now does two plain, static lookups instead — a board's own
Cargo.tomland its chip's, both grepped for an.../arch/...pathdependency (a few boards, e.g.
hailand the apollo3 boards, depend onan arch crate directly and via their chip, so both are checked). Any
other arch crate one of those depends on in turn (e.g.
cortexm4foncortexv7moncortexm) is compiled — and so checked — as a side effectof building it, without needing its own separate pick. The script now
lives in
tools/build/next to the siblinglist_boards.sh/list_archs.sh/list_chips.shit mirrors, is plain bash (nocargo metadata, noPython), and its name/comments say nothing about clippy — enumerating
boards per arch is all it does, and clippy is just one possible consumer.
Previously every board's clippy invocation unconditionally passed
-Zjson-target-spec, an unstable flag actually needed only byqemu_i486_q35 (whose target is a bare JSON filename rather than a path).
Fixed at the source instead: qemu_i486_q35's own
.cargo/config.tomlnow sets
[unstable] json-target-spec = true, the same persistent-configmechanism
../cargo/riscv_flags.tomlalready uses for the RISC-V boards'JSON specs (those are referenced by relative path instead, so didn't need
it spelled out explicitly). A plain
cargo clippy/check/buildnow allwork with no
-Zflag anywhere, so also dropped qemu_i486_q35's ownnow-redundant
CARGO = cargo -Zjson-target-specMakefile override.ci-job-clippystays a single target that loops over the board list(matching the existing style of
ci-job-archs/ci-job-chips), rather thanbeing split into per-board sub-targets — that split is only useful for
make -jdispatch, which is #2's concern, not this PR's.I looked for other Makefile lists with this same "static subset silently
standing in for full coverage" shape (per-arch/chip smoke-test lists,
QEMU/hardware test lists, etc.) and didn't find another instance worth
changing: everywhere else that needs "all boards/archs/chips" already
uses the existing
tools/build/list_*.shscripts, and the remaininghand-picked single-board jobs (
ci-job-msrv→ hail,ci-job-debug-support-targets→ nrf52dk) are explicitly arbitrarysingle-board smoke tests, not coverage claims.
Based directly on master; #2 (parallel
prepushdispatch) is rebased ontop of this one, since it needs the dynamic board list here to generate
its per-board sub-targets from.
Testing Strategy
make prepushpasses on a cold cache (make cleanfirst), including afull clippy run across the workspace + the 7 selected boards.
make -C boards/qemu_i486_q35still produces a working kernel.elf/.binafterremoving its
CARGOoverride, confirming the persistent-config fix fullyreplaces the per-invocation
-Zflag for every build path (not justclippy). Directly reproduced the old bare-filename-target error
(
`.json` target specs require -Zjson-target-spec) to confirm theflag was genuinely needed before the config fix, and that a plain
../cargo/*.jsonrelative-path target (what the RISC-V boards use)doesn't need it, matching
riscv_flags.toml's existing approach.A cold-context review caught a real (if currently harmless) bug in
list_arch_boards.sh's regex: it required exactpath = "..."spacing,but
chips/psoc62xa/Cargo.tomlandchips/rp2040/Cargo.tomlboth writepath="..."with none, so theircortex-m0pdependency silently didn'tmatch. Only harmless today because every board using those chips also
redeclares the same arch crate directly in its own
Cargo.toml— afuture cleanup removing that "redundant" direct dependency would
silently drop
cortex-m0pfrom coverage with nothing to flag it.Loosened the regex to tolerate any whitespace around
=, verifiedagainst both files. Also quoted
$(CURDIR)in the Makefile's cd-back(a checkout path with spaces would otherwise break it) and added a
missing
tput sgr0reset after each per-board banner'stput bold.The same review suggested cross-checking the result against
arch/'sdirectory listing to catch this class of gap generally. Not done:
several arch crates (
cortex-m3among them) are legitimately unused byany board today, so a blind diff would permanently false-flag those —
telling "unused" apart from "silently missed" again needs the full
dependency graph this script deliberately doesn't build.
TODO or Help Wanted
None.
Checklist
make prepush.PR Contents
Documentation