Status: foundation phases (0–3 + Phase 5 framework_libs delegation) landed in PR #207. Phase 6 acceptance gates and Phase 8.a
lib-selectCLI landed in PR #208. Phase 4 (zccache memoization,resolve_cached) shipped in PR #212 and is wired into the teensy + stm32 orchestrators by #214. Phase 7 perf gates remain a follow-up in#205.
PlatformIO's LDF picks the right libraries for a sketch but is slow (Python
- single-threaded + SCons graph overhead). fbuild's earlier basename-matching
helper produced wrong selections under #202 (STM32 SPI not auto-discovered)
and #204 (teensyLC RAM overflow from FNET / Snooze / RadioHead / mbedtls
being wrongly compiled). This subsystem replaces that helper with a
PlatformIO-LDF-faithful, Rust-native, deterministic resolver that orchestrators
call transparently through
fbuild-build::framework_libs.
Three crates form the subsystem:
fbuild-header-scan::scan— line-oriented C/C++ tokenizer that emitsIncludeRefper#include. Pure function, no I/O. Tracks comment, string-literal, raw-string, and char-literal state. Both branches of#if/#ifdefare scanned (false positives are acceptable, false negatives are not).fbuild-header-scan::walk— BFS over the include graph. Quoted-first resolution for"...", ordered search-path lookup for<...>. Visited set guards cycles. Output is canonicalized and sorted for deterministic cache keys.fbuild-library-select::resolve— PlatformIO-LDF-style two-pass walk:- From project seeds, BFS marks libs whose
include_dirscontain a reached path (path-prefix attribution). - Reconciliation: for each selected lib, re-walk seeded with its full
source set; libs newly reached in pass 2 are also marked.
Output
Selectionis sorted-by-name and deduplicated.
- From project seeds, BFS marks libs whose
fbuild-build::framework_libs is the integration glue — orchestrators
(teensy/orchestrator.rs, stm32/orchestrator.rs, ...) call
resolve_framework_library_sources transparently with no orchestrator-side
code changes.
project translation units framework libraries
(sketch, src/) (e.g. Arduino_Core_STM32/libraries/*)
│ │
│ collect_project_seeds │ FrameworkLibrary { name,
▼ │ include_dirs, source_files }
seeds: Vec<PathBuf> │
│ │
└────────────┬─────────────────┘
▼
fbuild-library-select::resolve
├─ pass 1: walk(seeds, project + lib include dirs)
│ └─ for each reached path:
│ attribute by include_dirs prefix → mark lib
├─ pass 2: for each marked lib, walk(lib.source_files, ...)
│ └─ newly reached paths attribute new libs
└─ Selection { included_files, required_libraries,
source_files, include_dirs, unresolved }
│
▼
fbuild-build::framework_libs
flatten Selection.source_files → Vec<PathBuf>
│
▼
orchestrator compile list
PlatformIO LDF's search_deps_recursive (piolib.py:428) attributes
resolved paths to libs by include_dirs prefix, not basename. fbuild does
the same. Concrete consequences:
- A project including
"foo/config.h"will not pull in aBarlibrary whosebar/config.hshares a basename. (Closed: misattribution risk.) - A library is selected only when the walker actually resolves an include
into its
include_dirs. (#204: FNET / Snooze / RadioHead / mbedtls no longer pulled in for a Blink sketch on teensyLC.) - STM32
SPI.hresolves throughArduino_Core_STM32/libraries/SPI/src/and prefix-attributes to the SPI library — no manual allowlist needed (#202).
Only project translation units seed the include walk. Headers under a local
lib/ tree are searched normally, but are never independent roots: a local
library header must be reached from the sketch's transitive include graph
before its framework dependencies can be selected. This prevents an inactive
header anywhere in a large library from self-selecting an unrelated framework
library (#1094).
The walk evaluates preprocessor branches, but only where it honestly can. The
macro set it is handed is the compiler command line — macros a header
#defines are not threaded through, because the walker visits each file once
in BFS order with a shared cache, and that is not preprocessor order.
So a guard is resolved three ways:
| guard | treatment |
|---|---|
| decidable from the command-line macros | evaluated; the dead arm is pruned |
references a macro the reachable corpus #defines somewhere |
undecidable — every arm is scanned |
| references a macro nothing defines | honestly false; the arm is pruned |
The middle row is #1371: FastLED derives FL_IS_SAMD21 several
headers deep from -D__SAMD21G18A__, so #if defined(FL_IS_SAMD21) is not
false — it is unknowable from the command line, and treating it as false hid
an #include <SPI.h> that genuinely compiles. The third row is what keeps
this from collapsing into a plain textual scan and reviving the
over-selection of #1094.
A literal #if 0 block is scanned as well. An include that can never compile
is there only to be seen: the PlatformIO LDF hint idiom, which FastLED uses in
platforms/*/ldf_headers.h. Its #defines are not applied, since that code
does not run.
Two separate comparisons with PlatformIO, since it is easy to conflate them:
- Against
chain, which evaluates no conditionals at all, fbuild is stricter: a guard the command line settles is settled here, and its dead arm is pruned. - Against
chain+, which does evaluate conditionals, fbuild is more permissive:chain+has no undecidable case, so a guard on a macro it cannot see reads as false — the exact failure this rule exists to avoid.
The #if 0 hint works under chain only as a side effect of it evaluating
nothing. chain+ does not honor it. fbuild honors it deliberately.
PlatformIO chain mode runs BFS from project sources, then ONE
reconciliation pass that re-seeds with each dependent lib's full source set
(piolib.py:1156). Unconverged deps drop silently (L1164–L1167). The
original issue framing ("fixed-point over include closure — typically 2–3
iterations") was wrong; we match PIO's 2-pass semantics exactly so users
who flip between PlatformIO and fbuild see the same library set.
resolve_cached (see crates/fbuild-library-select/src/cache.rs) hashes:
- sorted blake3s of project source content,
- sorted blake3s of each lib's canonical header
(
<include_dir>/<lib_name>.h), - ordered search-path list (order matters — PIO's resolution is order-sensitive),
- toolchain triple,
- framework install path + version identifier,
SCANNER_VERSION(bumped on tokenizer changes),LDF_MODE_VERSION(bumped on resolver semantic changes).
The KvStore is opened lazily by framework_libs::library_select_kv_store()
under fbuild_paths::get_cache_root().join("library-selection"), which
respects FBUILD_DEV_MODE and FBUILD_CACHE_DIR. Wiring the cache was a
pure addition: orchestrators that hit a backend error fall through to the
uncached resolve(...) rather than fail.
Walker output is sorted (BTreeSet → Vec). Resolver output is sorted by
lib name and deduplicated, and included_files, source_files, and
include_dirs are all sorted-and-deduped before return. Same inputs
produce byte-equal Selection output, which is what makes Phase 4 cache
keys safe.
- 34 scanner tests (
crates/fbuild-header-scan/src/scanner.rs) covering S-01..S-32 plus panic-safety guards for unterminated comments and strings. - Walker tests in
walker.rs(W-01..W-20: resolution order, cycle and diamond termination, deterministic output ordering, unresolved-include reporting). - Resolver tests in
crates/fbuild-library-select/src/lib.rsincluding the #204 regression guard, sort-stability, missing-include-dir tolerance, and same-basename-different-library disambiguation. - Acceptance gates for AC#1 (teensyLC) and AC#4 (stm32 SPI
auto-discovery) live in
crates/fbuild-build/tests/(teensylc_acceptance.rs,stm32_acceptance.rs). They are#[ignore]'d by default and run by CI with--ignored. They usefbuild-test-support'sElfProbeandCompileDbutilities to probe the produced firmware.
- Phase 7 — perf gates wired into
bench/fastled-examplesto catch regressions in both cold-resolve and warm cache-hit paths (#215). - Phase 8.b —
framework_libs.rsis now a thin delegator: scan-root collection, the uncached and cached entry points, theKvStoreopener, and tests. No remaining dead helpers to retire.