Skip to content

Evaluate FFTL CPU providers with explicit threading and plan ownership #590

Open

Description

@jtravs

What is missing

FFTL is a promising experimental CPU FFT provider, but it is not a drop-in replacement for either solprop-native's FFTW integration or the CPU implementation in #559. This issue records the feasibility investigation and the work needed before an adoption decision; it does not propose changing the default yet.

Inspected solprop e555fcf585b8014f36862422a7e92c7275f5e908, #559 at 1e609d68c2aa09e2f1b187cbf627a79826f5dd2a, and FFTL at e75616cddc75683684055f0038d046be753f68a3. Measurements were made on an Apple M5 Max over 8–9 November 2026.

Why it matters

FFTL's compiled C library can outperform FFTW on some relevant layouts, but using its FFTW-compatible names would silently defeat our external threading callback. The Julia path also needs different view handling and plan ownership to remain correct.

Native C++ integration

  • FFTL's compatibility header explicitly says fftw_threads_set_callback is accepted and ignored. Our callback detection checks only whether the call compiles, so it would incorrectly report scoped OpenMP-team support.
  • libfftl uses an embedded Julia pool, configured at build time; auto chooses a count at load time. fftw_plan_with_nthreads chooses plan task counts within that pool. It does not make those workers Kokkos/OpenMP workers.
  • FFTL omits fftw_version, which our Raman provider selection, tests and wheel smoke check use. A minimal compilation of that expression against FFTL's header fails.
  • MEASURE/PATIENT are ignored and wisdom is a placeholder, so our seeding and persisted-wisdom policy need provider-specific handling.
  • Threaded FFTL execution creates Julia tasks. Its C wrapper also creates another scratch-plan slot on first concurrent reuse. An allocation/lifetime audit is necessary before claiming our hot-path contract.
  • A local macOS bundle was about 26 MiB. Linux/aarch64 and Windows packaging, embedded-runtime coexistence, signals, foreign-thread entry and shutdown were not qualified.

Julia experiment

#559's blocked planner and transform_all! already use single-threaded FFTW plans inside Polyester. Preserve that arrangement for an FFTL experiment.

Actual probes on Julia 1.12.6 and 1.13.3 found:

  • mul! with FFTL rfft/brfft plans and the column-block SubArrays used by the experiment raises MethodError.
  • Complex-plan execution on a 10240×4 view allocates 1,310,880 bytes, whereas FFTW's view execution allocates zero.
  • Sharing one FFTL real plan across three simultaneous Polyester jobs failed all 20 rounds, with order-one or larger errors. FFTW passed. Separate FFTL plans restored relative max error below 5e-16.
  • Merely adding FFTL while continuing to call AbstractFFTs generic plans can still select FFTW for strided Float32/Float64 arrays.
  • FFTL has no UNALIGNED or get_num_threads bindings corresponding to those used here.

A possible adapter is explicit FFTL dispatch, one plan per concurrent stream and block, and its specialized pass APIs. Probed on Julia 1.13.3: plan_r2c_pass and plan_c2r_pass handle unit- and nonunit-stride views accurately, but allocated 576 bytes per tested call. plan_pass handled the unit-stride complex view without allocation; a nonunit first-dimension stride was refused. These are feasibility probes, not completed integration.

Reproduction and measurements

Build the pinned FFTL checkout with each Julia executable selected on PATH:

FFTL_THREADS=6 bash libfftl/build.sh
bash libfftl/test/fftw3_ctest.sh

With JuliaC 0.3.12:

  • Julia 1.12.6: six failures in each libfftl C test variant, including threaded 40×48×64 c2c, 30×64×100 real round trips, and 33×1024 agreement between one and four threads; 12 of 3,940 differential comparisons failed. The genuine FFTW control passed. A separate 33×1024 C reproducer gave relative max error 0.967. The ordinary Julia FFTL/RPlan versions of these shapes passed, so this is a compiled-library/build-specific observation, not a claim about every FFTL execution.
  • Julia 1.13.3: all C test variants and all 3,940 differential comparisons passed; the minimal reproducer passed. The six-thread performance problem also disappeared. The underlying compiler/runtime cause was not diagnosed.

C timing below is FFTL time / FFTW time; lower than one is faster. Double precision, forward-plus-backward pairs, nine interleaved batches, warm plans, Homebrew FFTW 3.3.11 with OpenMP callback, libfftl built using Julia 1.13.3 and apple-m1 target. Serial rows and transverse planes used FFTW MEASURE; threaded rows used ESTIMATE, matching native's effort policy. OpenMP block time was 1 ms. These are transform probes, not propagation runs.

shape 1 thread 6 threads 6 threads with an OpenMP array pass before each FFT pair
real 10240 × 4 rows 0.94 1.02 1.35
real 18432 × 17 rows 0.75 0.60 0.85
real 73728 × 17 rows 1.49 0.90 0.91
complex 8192 × 4 rows 0.72 2.23 1.99
complex 65536 × 1 row 0.74 0.56 0.81
complex 192 × 192 × 16 planes 0.60 0.63 0.78

All these C transform probes met a preselected 1e-11 relative-max envelope; measured forward differences were below 5e-16 and scaled round-trip errors below 1.4e-15. FFTW invoked the registered callback repeatedly; FFTL invoked it zero times. The OpenMP-pass experiment is synthetic and does not reproduce a full source evaluation. No end-to-end solprop speedup is established.

To do

  • Report or resolve the Julia 1.12.6 compiled-library failure upstream, and pin a qualified compiler/runtime toolchain for further C experiments.
  • If pursuing native: make provider capabilities explicit, remove false callback detection, handle missing version symbols and wisdom, and decide how one CPU pool will own execution.
  • If pursuing Julia backend experiment (do not merge) #559 first: add CPU-only FFTL dispatch, per-stream/per-block plan ownership, allocation-safe view execution and meaningful runtime metadata; retain GPU provider dispatch.
  • Qualify short field/envelope, Raman/plasma and transverse cases against numpy_ref, including concurrency and allocation checks.
  • Measure interleaved benchmarks ab short windows at representative widths on Apple silicon and an x86 CPU. Preserve current physics, normalisation and error bounds.
  • Qualify supported distribution platforms and runtime lifecycle before considering native adoption.

Done when

A follow-up records passing correctness/concurrency/allocation checks and short end-to-end A/B windows, or a documented decision to defer based on those results. Needs a decision: maintainers approve any new dependency and runtime/thread ownership design. Recommendation from this investigation: keep FFTW as the native default; prototype an explicit FFTL provider on #559 first.

Activity

  1. added
    enhancementNew feature or request
    performanceSlower or larger than it should be, with no wrong result
    area: nativeThe native backend, threading and FFT planning
    on Nov 9, 2026
  2. jtravs commented on Nov 12, 2026

    @jtravs
    ContributorAuthor

    The Julia/Polyester half is moot after ADR-077 (#763); the native C++ FFTL findings (threads callback ignored, no fftw_version, wisdom/MEASURE ignored, packaging) stand for any custom-FFT decision.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: nativeThe native backend, threading and FFT planningenhancementNew feature or requestperformanceSlower or larger than it should be, with no wrong resultpriority: lowWorth doing, no hurry

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions