feat(BMT): coupled Tier-3 donor linearization for the 2M+P3 ManualJacobian by haakon-e · Pull Request #744 · CliMA/CloudMicrophysics.jl

haakon-e · 2026-06-23T00:06:18Z

Make the ManualJacobian usable in mixed-phase by donor-linearizing the mixed-phase quadrature transfers into the substep Jacobian. For a transfer of primal rate S from donor d to receiver r, add -D on the donor diagonal and +D on the (r, d) off-diagonal with D = S/max(floor, x_d), reusing the per-process rates of _per_process_2mp3 (no gamma_inc shape derivative).

The dominant coupling is the ice->rain melt source (mass donor q_ice, number donor n_ice, rim mass/volume on their own donors); without it the rain melt source integrates explicitly and diverges with the step. The liquid-ice collision cloud/rain sinks self-limit on their own donors.

Purpose

To-do

Content

I have read and checked the items on the review checklist.

haakon-e · 2026-06-23T00:06:19Z

This change is part of the following stack:

feat(P3): Gauss-Legendre quadrature, integrand breakpoints -> GL(6) default #741
- feat(Rosenbrock): unified RosenbrockAverage substepping framework + 2M+P3 exact-AD mode #726
  - feat(Rosenbrock): 1M donor / coupled / exact modes on the unified framework #731
    - feat(Rosenbrock): post-solve per-process verbose diagnostics #732
      - fix(Rosenbrock): saturation adjustment on max(S_ice, S_liq) #737
        
        perf(BMT): recover the primal tendency from the Rosenbrock Jacobian pass #742
        
        feat(BMT): analytic ManualJacobian for the 2M+P3 Rosenbrock substep #743
        
        feat(BMT): coupled Tier-3 donor linearization for the 2M+P3 ManualJacobian #744 ◀

_{Change managed by git-spice.}

codecov · 2026-06-24T22:34:04Z

Codecov Report

❌ Patch coverage is 96.56652% with 24 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (he/rosenbrock-manual-2mp3@9b31858). Learn more about missing BASE report.

Additional details and impacted files

@@                     Coverage Diff                      @@
##             he/rosenbrock-manual-2mp3     #744   +/-   ##
============================================================
  Coverage                             ?   93.30%           
============================================================
  Files                                ?       57           
  Lines                                ?     3463           
  Branches                             ?        0           
============================================================
  Hits                                 ?     3231           
  Misses                               ?      232           
  Partials                             ?        0

Components	Coverage Δ
src	`93.97% <0.00%> (?)`
ext	`69.47% <0.00%> (?)`

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ref

The collision_cross_section_ice_ice docstring references [`ice_self_collection`](@ref), but the binding was absent from every @docs block, so Documenter could not resolve the cross-reference and the docs build failed. Add P3Scheme.ice_self_collection to the collision @docs block in docs/src/API.md.

…D substepping Introduce the RosenbrockAverage{Jacobian, GrowthTreatment, TendencyLimiter} tendency-mode framework and its two-moment + P3 exact-AD substep path. - RosenbrockAverage struct, keyword constructor, and the rosenbrock_exact() preset (ExactJacobian + ExplicitGrowthDiagonal + EndStateSaturationAdjustment). - The Jacobian / GrowthTreatment / TendencyLimiter option families with the concrete subtypes used by the exact preset. - The 2M+P3 linearized-implicit (Rosenbrock-Euler) driver differentiating the raw instantaneous tendency with ForwardDiff, the channel projection, the equilibrated solve, and the ice-only end-state saturation adjustment. - Documentation page (RosenbrockNumerics.md), API entries, and a 2M+P3 ExactJacobian framework test.

…fields Add a ClimaCore test that broadcasts bulk_microphysics_tendencies (RosenbrockAverage exact Jacobian, 2M+P3) over column and extruded-sphere spaces, exercising the highest-level substepping entry point. Gated to Julia >= 1.12, matching the inference-depth limit of the existing 2M+P3 kernel tests.

The Rosenbrock/2M+P3 docstrings cross-reference internal helper functions (e.g. the substep and per-process functions) that are not part of the exported API and therefore have no `@docs` entry. Set `warnonly = [:cross_references]` so these `@ref`s warn rather than fail the documentation build.

…elpers Add a dedicated error method for RosenbrockAverage with warm-rain-only Microphysics2MParams, so the failure names the missing P3 ice parameters instead of misreporting a Jacobian-choice problem. Document the internal Rosenbrock substep helpers in an API.md docs block and drop the site-wide warnonly cross-reference relaxation. Note on the Jacobian abstract type which options have methods.

Add a Dual method for `gamma_inc_moment` that takes the value from the plain-`Real` method and assembles the partials from the closed-form integrand: `∂M/∂D₁ = -D₁ᵖ e^{-αD₁}`, `∂M/∂D₂ = D₂ᵖ e^{-αD₂}`, `∂M/∂α = -M(D₁, D₂, p+1, α)`. Differentiating the plain method traces ForwardDiff through the difference of two near-equal regularized incomplete gammas; on narrow subintervals this loses precision and can flip the sign of the derivative in Float32. The analytic rule avoids the differencing and leaves the primal values unchanged.

The regularised-ratio blend weight (`sgs_weight_function`, used by `rime_density` and `rime_mass_fraction`) evaluated `(1 + tanh(2 * atanh(u))) / 2` with `u = 1 - 2 * (1 - a)^(-1 / log2(1 - a_half))`. In the sigmoid transition band, `u` rounds to exactly `±1` in single precision (the half-ULP gap at `1` exceeds `2 * (1 - a)^p`), so `atanh(u)` returns `±Inf`. The composed `tanh` recovers a finite value, but the ForwardDiff derivative is `Inf * 0 = NaN`. The existing extreme-value guards (`a > 42 * a_half`, `4 * a < eps`) do not cover this band for Float32, where `42 * eps(Float32) ≈ 5e-6` overlaps the physical scale of the rime volume `b_rim`. Replace the branch with the algebraic identity `tanh(2 * atanh(u)) = 2u / (1 + u^2)`, giving `(u + 1)^2 / (2 * (1 + u^2))`. This removes the `atanh`/`tanh` saturation, is value-identical to within one ULP across the band (byte-identical where the value is already `0`/`0.5`/`1`), and has finite derivatives throughout. It unblocks the exact-AD 2M+P3 Jacobian at rimed pure-ice states, where `ρ_rim = q_rim / b_rim` previously produced NaN partials.

…allocation/inference test

… test The exact-AD 2M+P3 Rosenbrock substep is type-stable and allocation-free on all CI Julia versions; the previous `VERSION >= v"1.12"` guard hid a test-harness artifact, not a real allocation. The `_framework_exact_call` helper returned a closure that captured the element type `FT`; capturing a type in a closure boxes it on Julia < 1.12, which made both `@inferred` report a non-concrete return and `@allocated` count the boxed call. Return a materialized argument tuple instead of an `FT`-capturing closure, and measure allocations with an interpolated `BenchmarkTools` benchmark (as in `bench_press`), which reports the call's own allocations rather than the surrounding closure. The `@inferred`, JET `@test_opt`, and zero-allocation assertions then pass on Julia 1.10, 1.11, and 1.12, so the guard is removed.

Extend the unified RosenbrockAverage framework to the one-moment model: add the DonorCellJacobian and CoupledDonorJacobian providers and channel masks, the rosenbrock_donor() and rosenbrock_coupled() presets, and the Microphysics1Moment substep driver. LinearizedAverage now forwards to rosenbrock_donor(), the donor-cell configuration of the framework. The one-moment kernels are made ForwardDiff-able so rosenbrock_exact() runs on the 1M model. Documentation and Float64/Float32 tests cover the donor/coupled/exact presets, the keyword constructor, donor-equals- LinearizedAverage, and zero-allocation hot calls.

The 2M+P3 RosenbrockAverage entry point dispatches only on ExactJacobian (or ManualJacobian); the default RosenbrockAverage() selects DonorJacobian, which the guard rejects with an ArgumentError. Pass rosenbrock_exact() so the kernel exercises the intended path.

Extract the shared saturation bisection from the two EndStateSaturationAdjustment methods. Rename _jacobian_1m_relinearized to _jacobian_1m_coupled to match CoupledDonorJacobian and rosenbrock_coupled(). Parametrize the 1M convergence tests over the limiter-free presets and use named presets instead of the bare default constructor; drop a rationale comment covered by the AD-compatibility guide.

Cover rosenbrock_donor, rosenbrock_coupled, and rosenbrock_exact through the 1M bulk_microphysics_tendencies entry on the GPU backend.

Guard the Verbose substep Jacobian on state finiteness, matching the non-verbose loops' Euler fallback. Add an informative error for Verbose with a non-Exact Jacobian on the 2M+P3 scheme. Test that a limiter-free exact-Jacobian mode gives identical verbose and non-verbose nets.

EndStateSaturationAdjustment now keeps the latent-heated end state at or above saturation over its more-supersaturated phase, max(S_ice, S_liq), instead of over ice alone. Below freezing ice carries the larger supersaturation, so the criterion reduces to the previous ice cap and the cold deposition-instability cure is unchanged; above freezing it caps on liquid and removes the warm under-condensation an ice-only cap leaves behind. Add a documentation section deriving the criterion with a 0-D vapor-exchange-only illustration of the Wegener-Bergeron-Findeisen transfer, and a framework test for the warm (liquid) and cold (ice) caps.

Mirror the warm subtest in the cold block, so a regression that disables the ice-side limiter is caught. Drop markdown emphasis from the saturation-adjustment documentation.

The mixed-phase saturation section labelled the coarse-step instability a "deposition instability", but the positive Jacobian diagonal sits in the rime-mass row (riming self-gain); the pure-deposition diagonal is negative there. Rename the reference to "ice-growth instability" to match the section heading and the mechanism.

ForwardDiff's Jacobian pass already evaluates the primal tendency f as the dual .value and discards it, so a separate f = g(x) repeats a full (quadrature- dominated) tendency evaluation. Recover f from the same pass: seed all N partials of the state in a single call to g with static Duals, and read the values and partials directly into the same FieldVector species type as x (MicroState1M or MicroState2MP3). Applies to the 2M+P3 and the 1M ExactJacobian paths; the analytic Donor/CoupledDonor 1M Jacobians (no f by-product) keep their separate evaluation. The DiffResults dependency is dropped: DiffResults.JacobianResult's mutable result buffer heap-allocates on GPU, so the seeding is done directly with StaticArrays instead. Both paths go through the shared _tendency_and_jacobian(jacobian, g, x) method, removing the duplicated code in the 2M+P3 substep loop. Measured on MicroState2MP3 (N=8, quadrature-dominated) and MicroState1M (N=4): the single full-width call is ~31% faster than the previous two-call f = g(x); J = FD.jacobian(g, x) for 2M+P3, and ~39% faster for 1M, with zero allocations in both cases. f and J are bitwise identical to the two-call reference across representative states. The tendency and Jacobian are unchanged, the path stays type-stable, and it composes with an outer AD pass.

Add a manually-linearized Jacobian (`ManualJacobian` + `rosenbrock_manual()`) for the two-moment + P3 microphysics Rosenbrock substep, as an alternative to the ForwardDiff `ExactJacobian` (which is ~2.7x the tendency and dominates the substep cost). Tiered after the one-moment donor Jacobians: - Tier 1 (closed form): cloud condensation/evaporation and ice deposition/sublimation (including the ice-number sublimation channel), the number adjustments, and the F23 nucleation number channel -- the stiff autocatalytic supersaturation block, captured exactly (matches ForwardDiff to ~3e-12 in Float64). - Tier 2 (donor-diagonal): the warm-rain transfers and freezing, linearized in their donor species via `D = rate / max(q_min, q_donor)`. - Tier 3 (dropped from the Jacobian): the quadrature ice collision / aggregation / melt couplings, kept explicit in the tendency. This avoids any `gamma_inc` shape derivative. The default `ExactJacobian` path is unchanged (purely additive). 0 allocations, type-stable, all existing rosenbrock tests pass.

…ries Add a ManualJacobian smoke testset mirroring the rosenbrock_exact one and validate the closed-form Tier-1 derivatives against ForwardDiff of the primal functions they linearize, across their branch regimes. Unify the ExactJacobian and ManualJacobian 2M+P3 substep entries into one method. Share the ice number-adjustment bounds between the primal tendency and the manual Jacobian; rename q_cap/cap_is_ice to q_limit/limit_is_ice; name rosenbrock_manual() in the 2M+P3 mode error message.

…obian Make the ManualJacobian usable in mixed-phase by donor-linearizing the mixed-phase quadrature transfers into the substep Jacobian. For a transfer of primal rate S from donor d to receiver r, add -D on the donor diagonal and +D on the (r, d) off-diagonal with D = S/max(floor, x_d), reusing the per-process rates of _per_process_2mp3 (no gamma_inc shape derivative). The dominant coupling is the ice->rain melt source (mass donor q_ice, number donor n_ice, rim mass/volume on their own donors); without it the rain melt source integrates explicitly and diverges with the step. The liquid-ice collision cloud/rain sinks self-limit on their own donors. Mixed-phase error vs the fine explicit reference drops from 88-2420 (non-convergent) to 0.0157-1.54 and converges under nsub, matching ExactJacobian to the same order of magnitude; ice-only is unchanged (melt/collision inactive below freezing). 0-alloc, type-stable (F32+F64); existing rosenbrock tests pass.

…Jacobian Add the missing rain-mass donor term of the liquid-ice collision to _jacobian_2mp3_manual, completing the set the surrounding comment and the ManualJacobian docstring describe. In a riming-active state the exact (rai, rai) diagonal is -1.7 per second while the previous manual entry was -3.5e-4; with the term the manual entry is -1.4 and substep refinement is monotone. Add a manual-mode convergence regression test in that state, reuse the Tier-2 donor factors, and state the ice-side receivers' positivity-clamp bound.

haakon-e force-pushed the he/rosenbrock-manual-2mp3 branch from 6ab368f to 414fb48 Compare June 23, 2026 00:32

haakon-e force-pushed the he/rosenbrock-manual-tier3 branch from 0f05968 to c8ebd41 Compare June 23, 2026 00:32

haakon-e force-pushed the he/rosenbrock-manual-2mp3 branch from 414fb48 to 3f36245 Compare June 23, 2026 21:32

haakon-e force-pushed the he/rosenbrock-manual-tier3 branch from c8ebd41 to 70688b7 Compare June 23, 2026 21:32

haakon-e mentioned this pull request Jun 23, 2026

fix: units of Barklie Gokhale 1959 #739

Merged

haakon-e force-pushed the he/rosenbrock-manual-2mp3 branch from 3f36245 to 1a5a00d Compare June 23, 2026 22:26

haakon-e force-pushed the he/rosenbrock-manual-tier3 branch from 70688b7 to f7f0804 Compare June 23, 2026 22:26

haakon-e force-pushed the he/rosenbrock-manual-2mp3 branch from 1a5a00d to 53e48a3 Compare June 24, 2026 21:56

haakon-e force-pushed the he/rosenbrock-manual-tier3 branch from f7f0804 to 5dc3a29 Compare June 24, 2026 21:56

haakon-e force-pushed the he/rosenbrock-manual-2mp3 branch from 53e48a3 to a2f9619 Compare June 25, 2026 18:01

haakon-e force-pushed the he/rosenbrock-manual-tier3 branch from 5dc3a29 to 3913282 Compare June 25, 2026 18:01

haakon-e force-pushed the he/rosenbrock-manual-2mp3 branch from a2f9619 to 22b1886 Compare June 26, 2026 00:38

haakon-e force-pushed the he/rosenbrock-manual-tier3 branch from 3913282 to c8c9b3d Compare June 26, 2026 00:38

haakon-e force-pushed the he/rosenbrock-manual-2mp3 branch from 22b1886 to ac7f724 Compare June 26, 2026 01:22

haakon-e force-pushed the he/rosenbrock-manual-tier3 branch from c8c9b3d to 1d56e11 Compare June 26, 2026 01:22

haakon-e force-pushed the he/rosenbrock-manual-2mp3 branch from ac7f724 to 8bdc7f1 Compare June 26, 2026 01:31

haakon-e force-pushed the he/rosenbrock-manual-tier3 branch from 1d56e11 to cdf374d Compare June 26, 2026 01:31

haakon-e force-pushed the he/rosenbrock-manual-2mp3 branch from 8bdc7f1 to cf9bec0 Compare July 1, 2026 05:35

haakon-e force-pushed the he/rosenbrock-manual-tier3 branch from cdf374d to cfa1630 Compare July 1, 2026 05:35

haakon-e force-pushed the he/rosenbrock-manual-2mp3 branch from a966e54 to 2228883 Compare July 11, 2026 00:58

haakon-e force-pushed the he/rosenbrock-manual-tier3 branch from 0ef8741 to db4e2f2 Compare July 11, 2026 00:58

haakon-e force-pushed the he/rosenbrock-manual-2mp3 branch from 2228883 to 58a62eb Compare July 11, 2026 02:14

haakon-e force-pushed the he/rosenbrock-manual-tier3 branch from db4e2f2 to df8a7ef Compare July 11, 2026 02:14

haakon-e added 23 commits July 10, 2026 19:25

docs(Rosenbrock): correct PR/doc claims and comments; add Rosenbrock …

d74c7aa

…allocation/inference test

test(BMT): run the 1M Rosenbrock modes in a device kernel

d0db917

Cover rosenbrock_donor, rosenbrock_coupled, and rosenbrock_exact through the 1M bulk_microphysics_tendencies entry on the GPU backend.

feat(Rosenbrock): post-solve per-process verbose diagnostics

6a0e180

test(Rosenbrock): show the unlimited cold end state violates saturation

a575b09

Mirror the warm subtest in the cold block, so a regression that disables the ice-side limiter is caught. Drop markdown emphasis from the saturation-adjustment documentation.

haakon-e force-pushed the he/rosenbrock-manual-2mp3 branch from 58a62eb to 958a343 Compare July 11, 2026 02:31

haakon-e force-pushed the he/rosenbrock-manual-tier3 branch from df8a7ef to 6f3f04c Compare July 11, 2026 02:31

haakon-e force-pushed the he/rosenbrock-manual-2mp3 branch from 958a343 to ea43e21 Compare July 11, 2026 03:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(BMT): coupled Tier-3 donor linearization for the 2M+P3 ManualJacobian#744

feat(BMT): coupled Tier-3 donor linearization for the 2M+P3 ManualJacobian#744
haakon-e wants to merge 36 commits into
he/rosenbrock-manual-2mp3from
he/rosenbrock-manual-tier3

haakon-e commented Jun 23, 2026 •

edited

Loading

Uh oh!

haakon-e commented Jun 23, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Jun 24, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

haakon-e commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

To-do

Content

Uh oh!

haakon-e commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Jun 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

haakon-e commented Jun 23, 2026 •

edited

Loading

haakon-e commented Jun 23, 2026 •

edited

Loading

codecov Bot commented Jun 24, 2026 •

edited

Loading