Repository navigation
Conversation
Add compile-time LIBXR_FASTCOPY_POLICY selection for Memory::FastCopy while keeping zero-length (including null pointers), self-copy, and 64-bit auto-vectorized paths intact: - SHIFTED_ODD (default): keeps 16-bit CopyAlignedWords<uint16_t> on even address differences and uses phase-specialized CopyShiftedLittleEndian<1/3> on 32-bit little-endian odd differences. - SHIFTED_ALL (driver/ch default): dispatches all 32-bit little-endian misaligned phases 1..3 through CopyShiftedLittleEndian<1..3>. - STD_MEMCPY (linux/webots/webasm and ESP Xtensa default): delegates to std::memcpy after zero-length and self-copy guards. - UNALIGNED_WORD (opt-in): direct 4x unaligned 32-bit word copy guarded by HAS_FAST_UNALIGNED_WORD static_assert.
Add phase/boundary regressions to test_mem.cpp and register 6 standalone test_mem_policy_* CTest targets covering SHIFTED_ODD (32/64-bit), SHIFTED_ALL (32/64-bit), STD_MEMCPY, and UNALIGNED_WORD across zero-length null pointers, self-copy, all 8x8 alignment pairs, sentinel guards, and overlapping forward/backward FastMove.
- Split LIBXR_FASTCOPY_POLICY into libxr_mem_policy.hpp and add independent LIBXR_FASTMOVE_POLICY, LIBXR_FASTSET_POLICY, and LIBXR_FASTCMP_POLICY knobs (BUILTIN/STD_LIBC) alongside LIBXR_MEM_WORD_SIZE
- Default driver/{linux,webots,webasm} to STD_LIBC for all four operations; default driver/esp Xtensa to Copy/Set=STD_LIBC while keeping Move/Cmp=BUILTIN per ESP32-S3 ROM vs builtin benchmarks
- Add loop-free FillSmallBytes<WORD_BYTES> for FastSet head/tail to prevent GCC -O3 -ftree-loop-distribute-patterns from emitting external memset calls
- Expand policy tests to 7 CTest matrix targets including mixed_esp_xtensa and update CMake options documentation in README.md
- Load all 8 words before writing back in CopyAlignedWords<WordT>'s burst loop - Eliminates GCC -O3 runtime alias-check fallback loops on Cortex-M3/M4/M7 - Restores ldrd/strd bursts for short forward-overlap FastMove (s - d < 32 B) and shrinks FastCopy+FastMove Flash footprint by ~332-336 B on ARMv7-M (and ~544 B on x86_64)
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Four commits on top of
dev(c84150262) introducing independent compile-time dispatch policies forMemory::{FastCopy,FastMove,FastSet,FastCmp}, 32-bit shift-and-combineFastCopy, a GCC-O3peel/tail fix forFastSet, and a load-before-store burst fix inCopyAlignedWords:333673853—perf(mem): add compile-time FastCopy policies and 32-bit shifted copyLIBXR_FASTCOPY_POLICYdispatch (SHIFTED_ODD,SHIFTED_ALL,STD_LIBC,UNALIGNED_WORD),CopyAlignedWords<WordT>,CopyShiftedLittleEndian<1U..3U>, andHAS_FAST_UNALIGNED_WORD-guardedUNALIGNED_WORD.fd6481d44—test(mem): add FastCopy policy matrix and boundary regressions(nullptr, nullptr, 0), non-overlappingFastMove, and 12×18 phase/boundary sweep coverage intest_mem.cpp, plus standalone policy matrix tests intest_mem_policy.cpp.ff150d818—perf(mem): add per-op memory policies and fix FastSet -O3 peelsrc/core/libxr_mem_policy.hpp(wrapped in/// \cond../// \endcond) and add independent per-operation policies +LIBXR_MEM_WORD_SIZE(4or8bytes):LIBXR_FASTCOPY_POLICY:SHIFTED_ODD(0, core default) /SHIFTED_ALL(1,driver/chdefault) /STD_LIBC(2) /UNALIGNED_WORD(3, opt-in, protected bystatic_asserton__ARM_FEATURE_UNALIGNED/ x86 / arm64).LIBXR_FASTMOVE_POLICY,LIBXR_FASTSET_POLICY,LIBXR_FASTCMP_POLICY:BUILTIN(0, core default) /STD_LIBC(1).cmake/config.cmake(FATAL_ERRORon invalid values; printsMemory policies: copy=..., move=..., set=..., cmp=...) and supportACTIVE_FASTCMP_POLICYinsrc/core/libxr_mem_cmp.cpp(keeping(size == 0U) || (a == b)guard underSTD_LIBC).driver/ch:Copy=SHIFTED_ALL,Move/Set/Cmp=BUILTIN.driver/espXtensa (CONFIG_IDF_TARGET_ARCH_XTENSAorIDF_TARGETinesp32/esp32s2/esp32s3):Copy=STD_LIBC,Set=STD_LIBC,Move=BUILTIN,Cmp=BUILTIN(ESP RISC-V inheritsCopy=SHIFTED_ODD,Move/Set/Cmp=BUILTIN).driver/{linux,webots,webasm}: all 4 default toSTD_LIBC.driver/{st,mspm0,hpm}&none:Copy=SHIFTED_ODD,Move/Set/Cmp=BUILTIN.-O3(-ftree-loop-distribute-patterns)FastSetregression insrc/core/libxr_mem_o3.cppby replacing tail/head< WORD_SIZEbyte loops with loop-freeFillSmallBytes<WORD_BYTES>, eliminating emitted externalmemsetcalls.README.mdand expandtest/automatic/CMakeLists.txtto 7-O3 -Wall -Wextra -Werrorpolicy matrix targets (includingtest_mem_policy_mixed_esp_xtensa).85d299af7—perf(mem): batch 8-word loads before stores in CopyAlignedWordsa0..a7) before writing back (dw[0..7]) inCopyAlignedWords<WordT>.-O3runtime alias-check fallback loops on Cortex-M3/M4/M7 (alias_cmp: 3 -> 0), restoresldrd/strdbursts for short forward-overlapFastMove(s - d < 32 B), preserves 4x unalignedldr.w/str.wmerge foruint16_ton ARMv7-M, and trims ARMv7-M Flash footprint by~332–336 B(~544 Bonx86_64).Behavior Changes
LibXR::Memory::{FastCopy,FastMove,FastSet,FastCmp}dispatch independently at compile time (if constexpr, zero runtime overhead) according toLIBXR_FAST*_POLICY.linux/webots/webasm) delegate all 4 functions tolibcby default; ESP Xtensa targets delegateFastCopyandFastSetwhile keepingFastMoveandFastCmpbuilt-in.nullptr), self-operation (dst == src/a == b), and overlap semantics remain unchanged.Verification (on
85d299af7)clang-format21.1.8&git diff --check: clean (cmake-format0.6.13verified on untouched CMake files sinceff150d818).doxygen1.12.0:933warnings (matchesdoc/doxygen_warning_baseline.txtbaseline933;0in touched files).Unix MakefilesDebug/ON, Release/ON, Release/OFF,NinjaDebug/ON,-DLIBXR_TEST_BUILD=ON):0compiler warnings across all 4 configs; all 4 pass16/16active tests (17total;test_linux_canskipped withoutvcan).LIBXR_SYSTEM=noneRelease builds with0warnings (Memory policies: copy=SHIFTED_ODD, move=BUILTIN, set=BUILTIN, cmp=BUILTIN).7/7test_mem_policy_*targets (shifted_odd_32,shifted_all_32,shifted_odd_64,shifted_all_64,std_libc,mixed_esp_xtensa,unaligned_word) pass with0findings..text + .literalFlash footprint vsdev(c84150262) (-O3/-O2 -ffunction-sections -Wall -Wextra -Werror,.rodata = 0 -> 0 B,0warnings):memsetcalls insideFastSetacross all 7 targets (m0p,m3,m4,m7,rv32,s3,x86), andalias_cmp = 0onm3/m4/m7withldrd/strdand unalignedldr.w/str.wmerge preserved:cortex-m0plus(SHIFTED_ODD + BUILTIN):2188 -> 2680 B(+492 B: Copy764->1248, Move896->884, Set240->260, Cmp288->288).cortex-m3(SHIFTED_ODD + BUILTIN):2140 -> 2432 B(+292 B: Copy872->1144, Move748->748, Set224->244, Cmp296->296).cortex-m4(SHIFTED_ODD + BUILTIN):2152 -> 2440 B(+288 B: Copy876->1144, Move752->752, Set224->244, Cmp300->300).cortex-m7(SHIFTED_ODD + BUILTIN):2136 -> 2424 B(+288 B: Copy876->1148, Move744->740, Set224->244, Cmp292->292).rv32imac(SHIFTED_ALL + BUILTIN):2128 -> 2512 B(+384 B: Copy772->1178, Move776->772, Set242->222, Cmp338->340).esp32s3(XtensaCopy/Set=STD_LIBC, Move/Cmp=BUILTIN):2091 -> 1223 B(-868 B: Copy698->39, Move828->821, Set226->24, Cmp339->339).x86_64(Linux allSTD_LIBC):4561 -> 102 B(-4459 B:25/25/25/27 B).UNALIGNED_WORD + BUILTINonm3/m4/m7:FastCopy 160 B, totals2140->1616 B (-524 B),2152->1624 B (-528 B),2136->1604 B (-532 B),alias_cmp = 0.ff150d818prior to85d299af7(unchanged policy logic):UNALIGNED_WORDstatic_assertpositive (m3,m7,x86) and negative (m0p,m3 -mno-unaligned-access,rv32,s3), plus Linux-DLIBXR_FASTCMP_POLICY=BUILTINand-DLIBXR_FASTSET_POLICY=BOGUSFATAL_ERROR.Silicon Benchmarks (prior commits
fd6481d44/ff150d818)x86_64baseline (FastCopy/FastMove/FastSet/FastCmpvslibc,596/596sweep checks pass):XR-DAP-S3@ 240 MHz): ROMmemcpybeatsFastCopy(0.59x–1.08x), ROMmemsetties/beatsFastSet(0.94x–1.10xaligned,0.43x–0.86xunaligned + saves202–211 BFlash), whereas ROMmemmoveis byte-by-byte (FastMove8.86x–19.12xfaster) and ROMmemcmploses at>=64 B(FastCmp1.74x–7.91xfaster oneq_0_0/eq_1_1), justifyingdriver/espXtensaCopy=STD_LIBC, Set=STD_LIBC, Move=BUILTIN, Cmp=BUILTIN.x86_64Linux: glibc AVX2 outperforms built-ins across all 4 primitives, justifyingdriver/{linux,webots,webasm}allSTD_LIBC.FastCopy(SHIFTED_ODD/SHIFTED_ALL/UNALIGNED_WORD): Cortex-M0+ (STM32G0B1, 1024 B odd)3.27x–3.43x; RV32IMAC (CH32V203, 1024 B misaligned)1.27x–3.41x; Cortex-M7 (STM32H723, 1024 B)1.34x–2.03x(DTCM) /1.34x–1.91x(AXI); Cortex-M3 (APM32F103CB) keeps(0,2)non-regressing (1565vs1552) underSHIFTED_ODD, with opt-inUNALIGNED_WORD12–64 Bin49–163cycles (1.30x–3.06xvsSHIFTED_ODD).FastSetFillSmallBytesfix re-benchmark (old -> new / libccycles & new ratio vsmemset,596/596sweep pass):off_078->73 / 152(2.082x)84->82 / 200(2.439x)100->102 / 776(7.608x)220->234 / 3080(13.162x)700->762 / 12296(16.136x)off_1129->106 / 152(1.434x, was 0.907x)135->109 / 200(1.835x)185->146 / 776(5.315x)305->278 / 3080(11.079x)785->806 / 12296(15.256x)off_086->79 / 118(1.494x)91->84 / 150(1.786x)110->114 / 534(4.684x)236->240 / 2070(8.625x)740->744 / 8214(11.040x)off_1142->113 / 118(1.044x, was 0.796x)147->118 / 150(1.271x)199->167 / 534(3.198x)320->290 / 2070(7.138x)800->770 / 8214(10.668x)off_043->46 / 42(0.913x)46->49 / 43(0.878x)58->61 / 59(0.967x)112->115 / 119(1.035x)328->331 / 359(1.085x)off_1126->54 / 55(1.019x)129->57 / 55(0.965x)156->84 / 68(0.810x)211->139 / 128(0.921x)427->355 / 368(1.037x)All tested boards were backed up via double-read and byte-identically restored to their intake flash state.