Skip to content

perf(cache): deliver hydrated core objects by reflink/link, not byte copy (#1414) - #1775

Merged
zackees merged 1 commit into
mainfrom
agent/core-cache-alias-hydrate
Oct 28, 2026
Merged

zackees merged 1 commit into
mainfrom
agent/core-cache-alias-hydrate

Conversation

@zackees

@zackees zackees commented Oct 28, 2026

Copy link
Copy Markdown
Member

What

FrameworkCoreCache::hydrate now delivers cached core objects with the cheapest
tier the volume supports — reflink, else hard link, else byte copy — instead
of fs::copy-ing every object into each new project directory.

This is item 3 of the perf list in #1414:

core-cache-hydrate byte-copies 118 objects, 1.17s [...] reflink_count=0, copy_count=2 on this filesystem, so the copies are not reflinked.

fbuild-build-engine already ships this exact tiering for zccache delivery
(zccache_embedded::probe_delivery_tiers / materialization_mode_for,
#1565) and already depends on reflink-copy, so this reuses the established
mechanism rather than adding a new one. Per-file escalation means no volume
probe is needed and an unsupported tier falls through to the next one
(EOPNOTSUPP for FICLONE on ext4, EXDEV for a link across volumes).

Scope / safety

  • hydrate may alias; store may not. FrameworkCoreCache::store still
    byte-copies (alias_ok: false), so the cache always owns its own bytes and an
    in-place write in some project's build directory can never corrupt a shared
    entry. A test pins this.
  • mtime is preserved on every tier. Incremental rebuild checks compare
    mtimes; a delivery that lost them would recompile the whole framework core,
    which is the cost the cache exists to avoid. Reflink stamps the source mtime
    explicitly; a hard link inherits it.
  • No change to cache keys, the summary lines, the CLI, or ArtifactCopyStats,
    so no log/CI assertions move.

Verification

New tests in framework_core_cache.rs (13 in module, all pass):

running 13 tests
test framework_core_cache::tests::alias_delivery_avoids_copying_bytes_and_keeps_the_mtime ... ok
test framework_core_cache::tests::delivery_without_aliasing_copies_and_stays_independent ... ok
test framework_core_cache::tests::store_owns_its_bytes_even_though_hydrate_aliases ... ok
test framework_core_cache::tests::hydrate_artifacts_alias_when_the_caller_allows_it ... ok
test result: ok. 13 passed; 0 failed; 0 ignored; 0 measured; 521 filtered out

Gates: bash test → 3937 passed, 47 suites, 0 failed; clippy -p fbuild-build-engine --all-targets -- -D warnings clean; ci.run_dylint --package fbuild-build-engine exit 0; enforce_platform_boundary.py --print-totals → rows=44; dylint_rows=43 (PB=0); check_dylint_allowlists.py
current; check_test_layout.py ok.

Measured cost of the delivery itself (118 objects × 100 KiB = 12.1 MB, ext4,
7 runs each, this host): byte copy 15.4 ms → FICLONE/hard-link 8.4 ms
(1.8×). Stated plainly because it is not the 1.17 s in the issue: that number
came from a host where each 100 KiB copy cost ~10 ms, which is a per-file-cost
regime (AV scanning / slow disk) rather than a bandwidth limit. In that regime
the aliasing tier removes the per-file read entirely, so the reported saving
should hold there; on a fast local volume the win is the ~7 ms measured above.
The change is free at all times and moves 12 MB less per fresh build directory.

Closes no issue on its own: #1414's remaining items (1 DWARF comp_dir
prefix map, 5 the ~930 ms unattributed tail) are untouched here.

#1414

…1414)

FrameworkCoreCache::hydrate copied all 118 framework core objects into every
new project directory. Deliver them with the cheapest tier the volume supports
instead: reflink, else hard link, else byte copy — the same mechanism the crate
already uses for zccache delivery, and the reflink-copy dependency it already
carries. store() keeps byte-copying so the cache owns its bytes; mtime is
preserved on every tier because incremental checks compare it.
@coderabbitai

coderabbitai Bot commented Oct 28, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@zackees
zackees added this pull request to the merge queue Oct 28, 2026
Merged via the queue into main with commit 8f26222 Oct 28, 2026
28 checks passed
@fastled-project-sync fastled-project-sync Bot moved this to Triage in FastLED Tracker Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Triage

Development

Successfully merging this pull request may close these issues.

1 participant