Repository navigation
Description
What is missing
FFTL is a promising experimental CPU FFT provider, but it is not a drop-in replacement for either solprop-native's FFTW integration or the CPU implementation in #559. This issue records the feasibility investigation and the work needed before an adoption decision; it does not propose changing the default yet.
Inspected solprop e555fcf585b8014f36862422a7e92c7275f5e908, #559 at 1e609d68c2aa09e2f1b187cbf627a79826f5dd2a, and FFTL at e75616cddc75683684055f0038d046be753f68a3. Measurements were made on an Apple M5 Max over 8–9 November 2026.
Why it matters
FFTL's compiled C library can outperform FFTW on some relevant layouts, but using its FFTW-compatible names would silently defeat our external threading callback. The Julia path also needs different view handling and plan ownership to remain correct.
Native C++ integration
- FFTL's compatibility header explicitly says
fftw_threads_set_callbackis accepted and ignored. Our callback detection checks only whether the call compiles, so it would incorrectly report scoped OpenMP-team support. - libfftl uses an embedded Julia pool, configured at build time;
autochooses a count at load time.fftw_plan_with_nthreadschooses plan task counts within that pool. It does not make those workers Kokkos/OpenMP workers. - FFTL omits
fftw_version, which our Raman provider selection, tests and wheel smoke check use. A minimal compilation of that expression against FFTL's header fails. - MEASURE/PATIENT are ignored and wisdom is a placeholder, so our seeding and persisted-wisdom policy need provider-specific handling.
- Threaded FFTL execution creates Julia tasks. Its C wrapper also creates another scratch-plan slot on first concurrent reuse. An allocation/lifetime audit is necessary before claiming our hot-path contract.
- A local macOS bundle was about 26 MiB. Linux/aarch64 and Windows packaging, embedded-runtime coexistence, signals, foreign-thread entry and shutdown were not qualified.
Julia experiment
#559's blocked planner and transform_all! already use single-threaded FFTW plans inside Polyester. Preserve that arrangement for an FFTL experiment.
Actual probes on Julia 1.12.6 and 1.13.3 found:
mul!with FFTL rfft/brfft plans and the column-block SubArrays used by the experiment raises MethodError.- Complex-plan execution on a 10240×4 view allocates 1,310,880 bytes, whereas FFTW's view execution allocates zero.
- Sharing one FFTL real plan across three simultaneous Polyester jobs failed all 20 rounds, with order-one or larger errors. FFTW passed. Separate FFTL plans restored relative max error below 5e-16.
- Merely adding FFTL while continuing to call AbstractFFTs generic plans can still select FFTW for strided Float32/Float64 arrays.
- FFTL has no
UNALIGNEDorget_num_threadsbindings corresponding to those used here.
A possible adapter is explicit FFTL dispatch, one plan per concurrent stream and block, and its specialized pass APIs. Probed on Julia 1.13.3: plan_r2c_pass and plan_c2r_pass handle unit- and nonunit-stride views accurately, but allocated 576 bytes per tested call. plan_pass handled the unit-stride complex view without allocation; a nonunit first-dimension stride was refused. These are feasibility probes, not completed integration.
Reproduction and measurements
Build the pinned FFTL checkout with each Julia executable selected on PATH:
FFTL_THREADS=6 bash libfftl/build.sh
bash libfftl/test/fftw3_ctest.shWith JuliaC 0.3.12:
- Julia 1.12.6: six failures in each libfftl C test variant, including threaded 40×48×64 c2c, 30×64×100 real round trips, and 33×1024 agreement between one and four threads; 12 of 3,940 differential comparisons failed. The genuine FFTW control passed. A separate 33×1024 C reproducer gave relative max error 0.967. The ordinary Julia FFTL/RPlan versions of these shapes passed, so this is a compiled-library/build-specific observation, not a claim about every FFTL execution.
- Julia 1.13.3: all C test variants and all 3,940 differential comparisons passed; the minimal reproducer passed. The six-thread performance problem also disappeared. The underlying compiler/runtime cause was not diagnosed.
C timing below is FFTL time / FFTW time; lower than one is faster. Double precision, forward-plus-backward pairs, nine interleaved batches, warm plans, Homebrew FFTW 3.3.11 with OpenMP callback, libfftl built using Julia 1.13.3 and apple-m1 target. Serial rows and transverse planes used FFTW MEASURE; threaded rows used ESTIMATE, matching native's effort policy. OpenMP block time was 1 ms. These are transform probes, not propagation runs.
| shape | 1 thread | 6 threads | 6 threads with an OpenMP array pass before each FFT pair |
|---|---|---|---|
| real 10240 × 4 rows | 0.94 | 1.02 | 1.35 |
| real 18432 × 17 rows | 0.75 | 0.60 | 0.85 |
| real 73728 × 17 rows | 1.49 | 0.90 | 0.91 |
| complex 8192 × 4 rows | 0.72 | 2.23 | 1.99 |
| complex 65536 × 1 row | 0.74 | 0.56 | 0.81 |
| complex 192 × 192 × 16 planes | 0.60 | 0.63 | 0.78 |
All these C transform probes met a preselected 1e-11 relative-max envelope; measured forward differences were below 5e-16 and scaled round-trip errors below 1.4e-15. FFTW invoked the registered callback repeatedly; FFTL invoked it zero times. The OpenMP-pass experiment is synthetic and does not reproduce a full source evaluation. No end-to-end solprop speedup is established.
To do
- Report or resolve the Julia 1.12.6 compiled-library failure upstream, and pin a qualified compiler/runtime toolchain for further C experiments.
- If pursuing native: make provider capabilities explicit, remove false callback detection, handle missing version symbols and wisdom, and decide how one CPU pool will own execution.
- If pursuing Julia backend experiment (do not merge) #559 first: add CPU-only FFTL dispatch, per-stream/per-block plan ownership, allocation-safe view execution and meaningful runtime metadata; retain GPU provider dispatch.
- Qualify short field/envelope, Raman/plasma and transverse cases against numpy_ref, including concurrency and allocation checks.
- Measure interleaved
benchmarks abshort windows at representative widths on Apple silicon and an x86 CPU. Preserve current physics, normalisation and error bounds. - Qualify supported distribution platforms and runtime lifecycle before considering native adoption.
Done when
A follow-up records passing correctness/concurrency/allocation checks and short end-to-end A/B windows, or a documented decision to defer based on those results. Needs a decision: maintainers approve any new dependency and runtime/thread ownership design. Recommendation from this investigation: keep FFTW as the native default; prototype an explicit FFTL provider on #559 first.
Activity
jtravs commentedon Nov 12, 2026
The Julia/Polyester half is moot after ADR-077 (#763); the native C++ FFTL findings (threads callback ignored, no fftw_version, wisdom/MEASURE ignored, packaging) stand for any custom-FFT decision.