perf: memoize successful path canonicalization (~35k realpath calls per build)

[zackees]

Problem

Part of #1537 (cause 3). ESP32-S3 builds spend ~0.7 s in work that no perf phase brackets, between the framework-libs cache and framework core cache log lines. It is cache-state independent — identical on warm and cold builds — which is why it never showed up as a phase.

Root cause

Building the framework-core cache key normalizes every path-bearing compiler flag of every core source. path_arg_for_compile_cwd (crates/fbuild-core/src/path.rs) canonicalizes both the path and the compile cwd on every call:

let stable_path = canonicalize_lexical(path).unwrap_or_else(|| path.to_path_buf());
let stable_cwd  = canonicalize_lexical(cwd).unwrap_or_else(|| strip_unc_prefix(cwd));

core_cache_key (crates/fbuild-build-engine/src/framework_core_cache.rs) calls compiler.artifact_cache_signature(...) once per core source, and the ESP32 compiler's signature includes the full include list. So per build:

385 include dirs x 46 core sources x 2 (path + cwd) = ~35,000 realpath(2) syscalls

The cwd is the same value in all 35,420 calls, and the include dirs are the same 385 for every source — ~386 distinct resolutions repeated 35,000 times, single-threaded, on the hot path of every build.

The same per-flag normalization also runs in refresh_command_hashes and needs_rebuild_with_signature, so this is not ESP32-specific.

Fix

Memoize canonicalize_lexical's successful realpath results only.

The parent-plus-filename fallback is deliberately not memoized: that is the branch that answers for paths that do not exist yet, so caching it would keep reporting a directory as missing for the rest of the process and a not-yet-created include dir would never resolve. A memo hit is therefore always a path that existed and was resolved once. The map is capped at 8192 entries so a long-lived daemon cannot grow it without bound.

Fixing it at canonicalize_lexical rather than at the ESP32 call site covers all the signature-building paths at once, rather than only the one that showed up in this profile.

Verification

Interleaved cold trials (median of 3, round-robined so machine drift hits every candidate equally, daemon torn down around each trial, and every trial confirmed a genuine framework-core cache miss):

build	wall	unphased	compile-core-variant
main	5.96s	709ms	2890ms
this PR	4.81s	218ms	2354ms

The framework-core cache key is unchanged (41b106e21c2b before and after) — which is the point: memoizing a pure function must not move the cache, and it does not.

The saving overlaps #1543 (fewer include dirs means fewer realpath calls too). On top of that change it is a further 3.97s → 3.79s, so this PR is worth landing independently of the order.

New tests pin both halves of the contract: a resolved path is memoized, and the missing-parent fallback is not (a directory created after the first call must still resolve correctly on the second).

• soldr cargo test -p fbuild-core --lib path:: — all pass
• Full unit suite: 3118 passed; sole failure is the pre-existing fbuild-python libpython3.13.so.1.0 loader error on this box, unrelated
• soldr cargo clippy --workspace --all-targets clean

Refs #1537

[coderabbitai]

Warning

Review limit reached

Next included review available in 53 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository: FastLED/fbuild/.coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 889e5230-a2cf-4879-9561-ef38f1507aef

📥 Commits

Reviewing files that changed from the base of the PR and between 1876518 and 6925b05.

📒 Files selected for processing (1)

• crates/fbuild-core/src/path.rs

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}