Repository navigation
fix: rsync --files-from link_stat failures on non-NFC filenames - #4
Merged
Merged
Conversation
scan_music_dir keyed its map by the NFC-normalized relative path and discarded the real on-disk path. run_rsync_device then handed that normalized key straight to rsync's --files-from, which does a literal link_stat against the filesystem — if the file is actually stored NFD (or any other non-NFC form), the stat misses and rsync fails every such file with link_stat ... No such file or directory (2) / exit 23. scan_music_dir now keeps the real on-disk relative path alongside (size, mtime), keyed by the same NFC comparison key so src/dst entries that differ only in normalization form still compare equal. to_copy is built and FAT-order-sorted by that key but emits the real path, which is what gets written to rsync's file list. remove_stale already did this correctly by resolving real paths from a fresh directory scan. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Addresses a non-blocking clippy::cloned_ref_to_slice_refs nit from review on 295213e. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What a user gets
Before: Syncing to a device whose library contains filenames that aren't
NFC-normalized (e.g. German audiobook chapter titles with ü/ä/ö/ß, stored
NFD-decomposed) fails for every such file:
Reported against the Mlove player syncing the hoerspiele library (Harry
Potter audiobook), but the bug affects any rsync-firmware device (sony,
rockbox, mlove) whenever a source filename isn't already NFC.
After: Files transfer regardless of Unicode normalization form. The
NFC-aware diff still treats NFC/NFD forms of the same name as equal (so it
doesn't re-copy every run), but rsync is now given the real on-disk path for
the actual transfer.
Root cause
scan_music_dir(src/engine.rs) keyed its map by the NFC-normalizedrelative path and discarded the real on-disk path.
run_rsync_devicethenhanded that normalized key straight to rsync's
--files-from, which does aliteral
link_statagainst the filesystem — a mismatch between thenormalized key and the real (e.g. NFD) on-disk bytes makes the stat fail.
remove_stalealready avoided this (it resolves real paths from a freshscan); this fix applies the same pattern to the copy side.
How it was verified
orch-lead): read the bug report, traced the root cause insrc/engine.rs, and reproduced the exactlink_stat/exit-23 failure witha minimal rsync repro (NFD-named file + NFC-normalized
--files-fromentry) before scoping the fix.
295213e): implemented the fix —scan_music_dirnowreturns
(real_path, size, mtime)keyed by the NFC comparison key;to_copysorts by that key (preserving FAT order) but emits the realpath. Added two tests: one on
scan_music_dir's map value, one thatshells out to real
rsyncand reproduces the failure with the old key /success with the real path.
cargo test: 36 passed, 1 pre-existingignored.
cargo build --release+cargo install --path . --force,installed-binary hash matched
target/release.dev-review): traced both call sites, confirmedremove_stale/to_deleteuntouched and still correct, independently rancargo test(36 passed) and
cargo clippy --all-targets(2 new cosmetic warnings inthe new tests, non-blocking). No blocking findings.
dev-qa): independent worktree build —cargo build,cargo test(36/36, including both new tests),
cargo build --release. Confirmed noorphaned callers of
scan_music_dir. Pass.11d63b3): addressed the reviewer's non-blocking clippynit (
std::slice::from_refinstead of cloning into a 1-element slice) inthe two new tests — test-only, no behavior change.
Not run: no physical device or the user's real
hoerspielelibrary onthis host (not present) — the real-
rsync-against-a-temp-NFD-fileintegration test is the closest available substitute, and it exercises the
exact
--files-fromcode path that failed in the report.What depends on this
This is the device-sync "diff → copy" step used by every rsync-firmware
device (sony, rockbox, mlove) — the fix benefits all of them, not only the
Mlove player in the original report. No UI changes.
Unsure
Nothing outstanding. The fix doesn't depend on knowing the exact non-NFC
form involved (NFD assumed, matching the reproduction) — "use the real path,
not the normalized one" holds regardless.
🤖 Generated with Claude Code