feat: CAM rename branch + establish what a CAM driver can reuse - #74
Open
aliakherati wants to merge 21 commits into
Open
feat: CAM rename branch + establish what a CAM driver can reuse#74aliakherati wants to merge 21 commits into
aliakherati wants to merge 21 commits into
Conversation
…ot just a shape
Topology already accepted nmodes=5, but nothing was registered, so "MAM5-JAX"
was a validated shape with no data behind it. This lands the data.
registered: ('cam_mam4', 'cam_mam5', 'e3sm_mam4_mom')
cam_mam5: nmodes=5 variant=cesm is_mam5=True pcnst=108
mode 5 = coarse_strat, so4 only, sigma_g 1.2, dgnum 9e-7
mam4_jax/core/cam_topologies.py is GENERATED in the sibling repo
mam-box-fortran and vendored here. That indirection is not laziness:
numptr_amode, lmassptr_amode and the per-slot species properties are not
tabulated in any source file -- modal_aero_data.F90 builds them at init from
the chemistry preprocessor's species list -- so they can only be observed,
not transcribed. The header carries the regeneration commands and a sha256
of each input dump.
THE DESIGN QUESTION, settled with data. CAM has no lspectype_amode: it
resolves each (mode, slot) straight to specdens_amode(l,m) and
spechygro(l,m), while Topology keeps per-TYPE tables plus an index. Rather
than give Topology two shapes, the generator synthesises the type list from
species-name prefixes -- legitimate only if lossless, so it was checked
rather than assumed. Across both topologies each of the 6 prefixes (so4,
pom, soa, bc, dst, ncl) carries exactly one (density, hygroscopicity) pair
in every mode it appears in. The generator reconstructs CAM's per-slot
arrays and exits non-zero on mismatch, and tests/test_cam_topology.py
re-checks it from the COMMITTED data, since a generator-time assertion says
nothing about what landed in the repo.
Full precision mattered. pom and bc both read 1.0e-10 hygroscopicity at one
decimal place but are 1.000000082740371e-10 and 1.000000013351432e-10 --
float32 round-trips of 1e-10 differing in the last bits. Had the dump stayed
at es12.5 the two types would have collapsed into one and the synthesis
would have been silently wrong. There is a test for it.
One assumption worth killing: MAM5 is NOT simply MAM4 plus a mode. The strat
size variant also changes accum's width, 1.8 -> 1.6. Asserted, because
"MAM4 + 1" is the natural intuition and it is wrong.
Also pins the numbers the upstream stale-dumfac bug report quotes, straight
from these tables: the last mode's dumfac gives +32.5% under MAM5 (last mode
is coarse_strat, sigma_g 1.2) against +20.6% under MAM4 (whose last mode
happens to share aitken's width). Those figures now have a test rather than
living only in a markdown file.
Registration deliberately does not activate: E3SM stays the default, so
importing the CAM tables cannot silently repoint existing code. Verified the
E3SM derived quantities stay bit-identical.
Suite: 176 passed (was 164; +12).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A3zu6nmLHsAUzTkDJFokjv
Before writing the driver, settle whether E3SM-derived kernels may be used
for CAM at all -- otherwise it is CAM's sequencing with E3SM's kernels.
Answered per process from the discrepancy reports plus one new numerical
check:
uptake rates reusable bit-identical, 58/58 normalised lines
condensation H2SO4 reusable VERIFIED numerically, worst 1.01e-14 over
200 random cases -- CAM's fgain + avg_uprt
formulation is algebraically the same
competing-sink system as the JAX closed form
with src = 0
coagulation reusable getcoags byte-for-byte identical, 1519 lines
nucleation reusable in a box: leaf parameterisations 0-diff, and
the one answer-changing difference is cloud
handling, which vanishes at cld = 0 (A4,
to be tested rather than assumed)
sulfate_equilib NOT CAM-only, no counterpart to have been ported
rename NOT CAM's A1 vs E3SM's C is 'a real algorithmic
difference' and the JAX port ports C
So exactly two items need new science work, and rename is the one that
matters most: it produced the x31 accumulation-number jump in the Fortran
runs, so a driver with the wrong one is qualitatively wrong, not
approximately right. Usefully, A1 (CAM default) and B (E3SM legacy) are
line-for-line identical -- 27 differing lines out of 243, none
answer-changing -- so it is one port serving both.
Records all 8 assumptions in a numbered table for individual accept/reject,
per owner request. A6 is flagged as a science-policy call needing sign-off:
sub-stepping defaults ON, diverging from CAM, because reproducing CAM's
un-substepped splitting faithfully means reproducing an O(50%) error at a
30 s step.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A3zu6nmLHsAUzTkDJFokjv
Found while establishing what a CAM driver may reuse. The docstring on
_mam_amicphys_1gridcell claimed that "calling this with a non-zero cloud
fraction in any cell raises a clear error so future workflows don't silently
get wrong physics". The code immediately below declined to check, and cldn
was then never read at all.
So a non-zero cloud fraction was silently ignored: only the clear sub-area
is ported, and the Fortran splits the cell and area-weights two different
sub-area calculations, so the port applied clear-sky physics to the WHOLE
cell and returned a plausible answer. Exactly the failure the docstring
claimed to prevent.
WHERE THE CHECK HAS TO LIVE. The stated reason for not checking was that
cldn may be traced -- true, and it matters more than it looks: driver.run_step
is @jax.jit, so on the normal path cldn is ALWAYS a tracer. A check inside it
could never fire. Putting one there and warning on the traced case would have
fired a warning on every single call, and a warning that always fires gets
filtered, at which point it protects nothing while still reading as
protection.
So the guard runs where the value is still concrete:
* _check_clear_sky raises for a concrete non-zero cldn, reporting the
magnitude it saw, and returns SILENTLY when traced.
* driver.run_step / run_timesteps are now thin UNJITTED wrappers that
validate before tracing; the jitted implementations moved to
_run_step_jit / _run_timesteps_jit. One Python call per step, no tracing
overhead, jit cache still on the inner functions.
* amicphys() checks too, for direct callers.
Verified no test depended on the public names being jit objects, and there is
now a test asserting they are NOT -- if either gets decorated again the guard
becomes unreachable, which is the bug this arrangement exists to avoid.
Twelve tests, including that any single non-zero cell refuses rather than
just a non-zero mean, and that the refusal reports the magnitude (a bare
refusal sends the reader hunting for which field was wrong).
Suite: 176 passed, and no longer emits a warning on every driver call.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A3zu6nmLHsAUzTkDJFokjv
The blocker for a CAM driver. CAM's default path
(modal_aero_rename_no_acc_crs_sub) and E3SM's production path
(mam_rename_1subarea, rename_method_optaa = 40) are a real algorithmic
difference, and this repo ports the latter -- so a CAM driver could not
reuse it. Rename is not a detail: it produced the x31 accumulation-number
jump in the Fortran box runs, so the wrong one is qualitatively wrong.
WHAT MADE THIS SMALL. E3SM's code CONTAINS CAM's algorithm as its
optaa /= 40 branch, and CAM's path is line-for-line identical to E3SM's
legacy rename -- 27 differing lines out of 243, none touching the
arithmetic. So this is a selector over three decisions, not a second
implementation, and every quantity the CAM branch needs (dryvol_t_del,
dp_cut_mfrm, the unrescaled dryvol_t_old) was already computed:
1. CAM gates up front on dryvol_t_del <= 1e-6*dryvol_t_oldbnd; E3SM
skips that and applies a different gate after rescaling.
2. CAM's clamp fires when the NEW diameter reaches dp_cut and clamps the
diameter only, via min(). E3SM's fires on the OLD diameter exceeding
dp_belowcut.
3. CAM leaves the old VOLUME unrescaled; E3SM rescales it by
(dp_belowcut/dgn_t_old)**3.
Default stays "e3sm", so the production path and its Fortran comparison are
bit-unchanged -- asserted, not assumed.
VALIDATION, and its limits. The captured reference fixture turns out to be
unable to tell the branches apart: at that state the transfer barely engages
and both agree exactly. A suite built only on it would have concluded the
CAM branch was a no-op, so there is now a test pinning that fact.
Oversizing the aitken mode reaches the regime where they diverge, and the
discriminating test is CAM's growth gate: with zero growth CAM transfers
NOTHING while E3SM still moves 7.69e7 particles. That matches the Fortran
capture, where growth 0 gives a zero transfer.
Also tested: both branches conserve number and mass exactly, and CAM's
transfer grows then SATURATES with the growth driver -- the tail beyond the
boundary is finite, and a port that scaled without bound would be wrong in
a way conservation and positivity cannot detect.
One correction worth recording: the conservation baseline is qaer_cur ALONE,
not qaer_cur + qaer_delsub_grow4rnam. Growth is already folded into
qaer_cur and the delta is informational, used only to form dryvol_t_del.
The Fortran reference conserves against qaer_cur to 0.0 and against
qaer_cur + delta to 2.4e-3, so the wrong baseline reads as a 0.24% leak --
which is what I first reported to myself before checking.
NOT CLAIMED: numeric agreement with the Fortran. That needs the unit and
index mapping between CAM's q/dqdt convention and the amicphys-local view.
A reference set exists (mam-box-fortran tools/capture_rename), and the test
module says plainly that the comparison is the next step.
Suite: 184 passed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A3zu6nmLHsAUzTkDJFokjv
…claim Numeric validation of the CAM rename branch, done WITHOUT mapping CAM's q/dqdt convention into the amicphys-local view. That mapping is where a factor-of-1000 error produces a FAKE validation -- failing for the wrong reason, or passing because two errors cancel. Instead the comparison is on dimensionless quantities. Both inputs (v2n/voltonumb, deldryvol/dryvol) and outputs (xferfrac_num, xferfrac_vol) are unit-free, so no conversion appears anywhere. Worst relative difference across five growth values: 9.6e-10, the floor the reference's 9 significant figures can resolve. BUT THE SCOPE IS NARROWER THAN THAT NUMBER SUGGESTS, and a guard test I wrote to check the comparison was actually discriminating is what caught it. At frac_v2nzz = 0.3 the two branches agree EXACTLY. So the five points validate the machinery they SHARE -- the erfc tail integrals, dp_cut, factoraa/factoryy, the transfer-fraction clamps -- and not CAM's three specific decisions. They cannot diverge there: num_t_oldbnd is clamped into [dryvol*v2nhirlx, dryvol*v2nlorlx], so dgn_t_old saturates however far the aitken number is pushed, and both paths cap at the same fraction. Sweeping frac_v2nzz from 1.0 to 0.001 gives 4.92875520e-01 for both, identically. The branches DO differ -- on the captured reference state, zero growth gives CAM 0 transfer against E3SM 7.69e7 particles. What is missing is a Fortran capture in a DIVERGENT regime, which is now item 1 of the remaining work. My first version of that guard asserted the E3SM branch would NOT reproduce CAM's fractions. It failed, correctly, and rather than weaken it into something vacuous it is rewritten to assert and document the true state: both branches reproduce the Fortran at these points, and here is why, and here is what that does not establish. Plan 025 updated with progress and three new assumptions (A9-A11), including that the five capture points are probably NOT representative since they all sit in the saturated regime. Suite: 190 passed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A3zu6nmLHsAUzTkDJFokjv
…erges The earlier comparison landed in a regime where both algorithms agree, so it validated shared machinery rather than CAM's three decisions. This closes that gap. FINDING THE DIVERGENT WINDOW required deriving it rather than searching: E3SM clamps when dgn_t_old > dp_belowcut (= 0.99*dp_cut) CAM clamps when dgn_t_new >= dp_cut and dgn_t_new = dgn_t_old*(1+growth)^(1/3), so divergence needs an OVERSIZED mode with SMALL growth -- dgn_t_old just above dp_belowcut while dgn_t_new stays below dp_cut, which bounds growth under about 3%. Confirmed: at dgn_old = 8.12e-8 the branches diverge at growth 0.005 and 0.01 and then CONVERGE AGAIN at 0.03, exactly where the derivation says they should. RESULT, six points across two diameters and three growth values: JAX cam branch vs CAM Fortran worst 8.1e-15 (machine precision) JAX e3sm branch vs CAM Fortran off by 68% to 308% So the comparison genuinely discriminates -- a method="cam" that silently fell through to the E3SM path would fail by orders of magnitude, and there is a test asserting exactly that. Getting the reference needed a units fix in the capture tool (committed in mam-box-fortran): rename's dryvol is per kmol-air, q*(specmw/specdens), so parameterising by raw volume put v2n below the v2nhirlx floor and clamped every input to the same bound. Two different states produced byte-identical output with no error anywhere. Suite: 202 passed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A3zu6nmLHsAUzTkDJFokjv
The CAM rename branch is now validated to 8.1e-15 against CAM's Fortran in the regime where it differs from E3SM by 68-308%. The divergent window was derived from the two clamp conditions rather than searched for, and the derivation was confirmed behaviourally: the branches diverge at growth 0.005 and 0.01 and converge again at 0.03, exactly where the algebra says. Corrects assumption A10, which I had recorded as 'probably not representative' -- it was in fact wrong. All five original capture points sat in the saturated regime where num_t_oldbnd clamps and both branches agree. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A3zu6nmLHsAUzTkDJFokjv
There were 39 committed figures for the E3SM validation work and none for
the CAM port, so the overnight results existed only as tables in commit
messages. Three panels, each answering a question that actually came up:
1. Rename: how far apart are the algorithms, and where? Up to 4.3x, and
only inside a window -- they agree at small growth (CAM's gate has not
opened) and converge again once growth pushes dgn_new past dp_cut.
2. Rename: is the CAM branch right? The Fortran reference points sit on
the CAM curve while the E3SM curve is a factor of several away.
3. Condensation: 400 random cases of CAM's fgain + avg_uprt formulation
against the JAX closed form, clustered at 1 machine epsilon.
Two plotting notes worth recording, since both fail silently:
* annotate(xy=(0, 0)) draws NOTHING on a log axis -- (0,0) is not a valid
data point. Four labels vanished with no warning and only turned up on
inspecting the rendered PNG. Pure text labels now use text() with the
axes transform.
* The "up to 4x" in the title was written before measuring. The actual
maximum ratio is 4.31 at dgn_old = 1.2e-7, so the claim happened to
hold, but it was a guess -- it is now computed and the title says 4.3x.
Palette is the validated four-colour categorical set (validate_palette.js,
light mode, all checks pass).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A3zu6nmLHsAUzTkDJFokjv
…t/cam-driver Semantic resolution beyond the textual conflicts: - driver.py keeps this branch's structure — jitted _run_step_jit / _run_timesteps_jit inner functions behind unjitted public wrappers whose only job is the pre-trace clear-sky check — and threads main's params (AmicphysParams pytree) and static mdo_pcarbonaging through both layers. The scan body calls _run_step_jit directly (the guard cannot read a tracer anyway). - amicphys() combines both sides: _check_clear_sky at the public entry (this branch) before the params-default resolution (main), and _mam_amicphys_1gridcell takes params per main. - test_run_step_params_is_traced_not_static: the jit-cache bookkeeping (clear_cache/_cache_size) moves to _run_step_jit — run_step is now an unjitted wrapper. Full suite green (219) after the resolution.
…ine precision The sulfate_equilib work (plan 025 remaining item 1) splits into the equilibrium VALUE and its consumption by gasaerexch's reversible branch; this ports the value. mam4_jax/physics/strat_sulfate.py carries calc_h2so4_wtpct (Tabazadeh 1997 composition) and calc_h2so4_equilib_mixrat (Ayers/Kulmala vapor pressure, Giauque enthalpy, dual Kelvin factors) from modal_aero_wateruptake.F90:895-1171 (cam6_4_187), plus the CAM qsat_water they stand on. Decisions and findings: - CAM's qsat_water is NOT the already-ported E3SM box one: CAM returns qs = 1 whenever p <= es; the E3SM box clamps only a negative-denominator qs. They disagree on es in [p, p/(1-eps)] -- reachable at the routine's own t = 450 K clamp -- so the CAM cluster carries its own private helper. - The Fortran routines are PRIVATE to their module and unreachable by the box model's process masks; captured via a visibility-only patch applied to a staged copy by the new mam-box-fortran tools/capture_sulfeq (patch + tool + justification live there). - The capture grid pins every branch by construction: both T clamps, all three Tabazadeh activity regimes plus both activity clamps (qh2o is built as activ_target * qs so coverage cannot drift with T/p), Kelvin-strong to Kelvin-negligible diameters (1e-8 to 9e-7 m, the MAM5 coarse_strat dgnum). 27 qsat + 189 wtpct + 567 full cases. - Measured worst rel-err: wtpct 1.3e-15, sulden 7.1e-16, qh2so4_equilib 3.5e-14 (exp of a ~100-magnitude exponent amplifies its last ULP by ~1e-14; gated at 5e-13). Kelvin monotonicity in diameter and reverse-mode gradient finiteness locked in per branch. - Upstream defect found and preserved: the first surface-tension interpolation pairs knot i-1's ordinate with knot i's abscissa (F90:1005), offsetting the segment by -(sig2-sig1). Written up in mam-box-fortran docs/bugs/BUG-cam-wateruptake-surftens-interp.md; the port keeps bit-parity with the reference. Full suite green (224).
modal_aero_gasaerexch.F90:523-566, the stratospheric branch that replaces irreversible fgain-split condensation: the gas decays exponentially toward the mode-weighted equilibrium g_equ = sum(uptk*sulfeq)/sum(uptk), each mode takes dqdt = uptk*(g_avg - sulfeq_n), over-saturated modes EVAPORATE down to the a_end >= 0 floor. Faithful details carried: the kxt < 1e-5 first-order branch, the deltatxx = deltat*(1+1e-15) nudge, and the three-state ido_so4a classification (1 = mode has an so4 slot, 2 = CAM's slotless pcarbon age-source condensing with a_bgn = 0, 0 = inactive). One documented arithmetic deviation, repo-standard (plan 026 / ADR-019): 1 - exp(-kxt) is written -expm1(-kxt), worth ~2e-11 relative at the branch threshold and nothing anywhere else. Validation (tests/test_strat_sulfate.py): a verbatim NumPy transcription of the Fortran block (loops, 1-exp, cycles) over 300 randomized states spanning both g_avg branches, condensation and evaporation, the floor, and all ido classes, gated at 5e-11 (sized by the expm1 deviation); exact-zero equilibrium fixed point; floor-limited evaporation (dqdt = -a_bgn/deltatxx when the mode empties); branch continuity at kxt = 1e-5; reverse-mode gradient finiteness across every where(). The end-to-end driver comparison (plan 025 remaining item 1-2) will validate this against the actual Fortran box rather than a transcription. Suite: 9 strat_sulfate tests green (228 total).
…it consumes A6 (sub-stepping ON by default) approved by owner 2026-08-26; substep count to be picked empirically in G5. Records the source-read findings that reshape the reuse table: CAM's gas_aer_uptkrates is a third variant (fixed beta=2, ac=0.65 literals, truncated constants — port, don't reuse); the legacy 8.0-monolayer aging runs inside BOTH CAM's gasaerexch and coag_sub (closing the #75 attribution question — CAM genuinely ages at 8.0, amicphys at 3.0); DGNUM pbuf-inits to 0.0 so end-to-end parity needs topology-threaded calcsize+wateruptake while the microphysics sequence validates against isolated captures first; MAM5 references pin nl_acc_crs=0 (rename-A2 stays deferred); merges feat/cam-mam5-topology (PR #73) into this branch as the driver's data layer.
… plan 025 G0 The CAM driver needs molecular weights the topologies don't carry: per-type specmw for fac_m2v in the legacy aging blocks, and the chemistry mechanism's adv_mass over the gas window for the mmr<->vmr boundary conversion. Both come from the chemistry preprocessor at init and are not tabulated in any source file — same provenance argument as the index tables — so the sibling repo's dump tool now emits SPECMW_BY_SLOT / ADV_MASS / CNST_NAMES / MWDRY read out of the initialised box model, and a new to_params.py generator (mirroring to_topology.py, including the lossless per-type synthesis check and a type-ORDER assertion against the topology synthesis) produces mam4_jax/core/cam_params.py. tests/test_cam_params.py re-checks from the committed data: alignment with the topologies, every lmassptr/numptr index landing on a tracer whose NAME carries the right species prefix and mode suffix (the strongest cross-file consistency check available), the plan 024 §3 census values (so4 115.10734, organics/bc 12.011, dst 135.064039, ncl 58.442468), and the mechanism gas MWs (H2SO4 98.0784, SO2 64.0648). cam_topologies.py is re-vendored with new source-sha stamps (the indices files gained fields); content verified identical.
coupling/cam_driver.py is the first real consumer of the Topology axis (tables resolved from the Topology ARGUMENT + cam_params, cached per name — never a module-global read inside a kernel). It carries: - gas_aer_uptkrates_cam: CAM's THIRD uptake-rate variant, ported verbatim — fixed beta = 2 (amicphys computes a Knudsen-dependent beta), hardcoded ac = 0.65 Fuchs-Sutugin literals (0.4875, 1.184), truncated tworootpi/root2/quadrature constants, CAM's own gasdiffus/gasspeed, result x number concentration. - modal_aero_gasaerexch_cam: fgain-split irreversible condensation (trop) / reversible sulfeq-limited solve (strat, the ported h2so4_reversible_uptake); the LEGACY pcarbon aging block at its own hardcoded 8.0 monolayers (gasaerexch.F90:37, deliberately not configurable and not shared with amicphys' 3.0); the condensed-on-pcarbon so4 redirect to accum gated on the aging actually firing (CAM drops it at xferfrac = 0); rename A1 (aitken->accum, acc_crs off) fed the accumulated dqdt; dense tendency application. - SO4-only reductions each exact for the scope, not approximate (docstring): no nh4/msa; SOA tracers identically zero (A13); qqcw identically zero; diagnostics not carried. Validation: tools/capture_gasaerexch (sibling repo) calls the REAL subroutine with prescribed diameters — no calcsize/wateruptake in the loop — over 24 cases x both topologies crossing trop/condensing-strat/ evaporating-strat, rename off/firing, saturated/fractional aging, and two gas loadings. Worst rel-err 1.1e-15 on everything that is not a cancellation sliver of its own input. The post-aging pcarbon number (a q*10eps remnant) differs by up to ~5% OF THE SLIVER (~1e-15 of the tracer): the reference binary FMA-contracts q + dqdt*deltat (gfortran -O2 on arm64), verified bit-for-bit by reproducing the two-rounding chain (= JAX value) and the single-rounded FMA (= Fortran value); the bar is rtol 5e-13 plus per-slot atol 1e-13*q_in, with transfer DESTINATIONS that start at zero compared strictly. A branch-coverage test asserts the grid genuinely fires rename, aging, and strat evaporation. A13 is confirmed empirically: the capture runs the full Fortran including soaexch on zero SOA state and parity holds. Full suite green (249).
The nucleation leafs are the already-validated ports (0-diff CAM vs E3SM); this adds what the CAM WRAPPER owns: - the step-average H2SO4 reconstruction from del_h2so4_gasprod / del_h2so4_aeruptk (pre-uptake gas, log-ratio uptake rate clamped at 20 with the q3-floor clamp, production-during-decay closed form); - relative humidity through CAM's GENERIC qsat — the mixed-phase LOOKUP TABLE (estblf: 250 one-kelvin entries, Goff-Gratch water above tmelt, Goff-Gratch ice below tmelt-20K, linear blend between), ported as physics/cam_saturation.py. This is NOT the direct over-water qsat_water the Tabazadeh cluster uses: at 232 K they differ by the full water-vs-ice SVP ratio, so using the wrong one would silently shift every cold-case nucleation rate; - the 4e-16 cutoffs on current AND average H2SO4, the 100 #/kmol/s rate floor, aitken size constraints, tendency application (cld = 0 in scope so the (1-cldx) weights are 1); - mer07_veh02_nuc_mosaic_1box gains an optional mw_so4a_host argument (the Fortran passes it as an argument for the same reason): None resolves to data.MW_SO4A_HOST as before — E3SM path bit-unchanged — and the CAM driver passes its topology's 115.10734. Validation: tools/capture_newnuc (sibling repo), 96 cases x both topologies crossing warm/cold (the table's ice branch), in-PBL vs free troposphere, cutoff-grazing to strongly nucleating H2SO4, production and prior-uptake on/off, two humidities. Worst rel-err 2.0e-11 — the ~1-ulp form differences (expm1 vs exp()-1, table interpolation) amplified through the ~10th-power H2SO4/RH sensitivity of the nucleation rate; gated at 1e-10. Sulfur closure (so4_ait gain == H2SO4 loss) exact per case; both cutoff directions exercised.
The getcoags kernel is byte-identical CAM-vs-E3SM and already ported;
this adds coag_sub's ORCHESTRATION on the gas window:
- three Whitby-coefficient pairs (ait->acc, pca->acc,
ait->pca-effective-accum) via getcoags_wrapper_f;
- sequential number solves, each consuming the PRIOR mode's
time-average number: accum implicit self-coag, then pcarbon and
aitken through the faithful three-branch closed form
(|tmpc| < 0.01 exponential, |tmpa| < 0.001 self-dominated, general
tmph form), all double-where guarded;
- the combined aitken mass transfer: losses to accum AND pcarbon move
in one step, all delivered to the ACCUM species (the
pcarbon-destined share is deemed aged through), with that share x
fac_m2v_aitage accumulating the aging shell volume — so4 at face
value, soa at its equivalent-so4 hygroscopicity factor
(spechygro_soa/spechygro_so4), ncl/dst contributing zero;
- the coag-side legacy 8-monolayer aging fraction ('this duplicates
the code in modal_aero_gasaerexch', and here it genuinely does);
- pcarbon mass at direct-coag + aging (capped 1-10eps), then the
aged number transfer pca->acc.
Constant correction found by the capture: shr_const_rgas is the
PRODUCT 6.02214e26 * 1.38065e-23 = 8314.467591 J/K/kmol. The rounded
8.31446e3 shortcut is 9.1e-7 relative off, and the implicit
coagulation number solves amplified that to ~2e-6 — the largest
error in the whole CAM chain until fixed. cam_driver now computes the
constant exactly as shr_const_mod does; gasaerexch parity also
improved (1.04e-15 worst).
Validation: tools/capture_coag (sibling repo), 32 cases x both
topologies — number loadings pushing all three closed-form branches,
saturated (zero-core) and fractional aging, aitken diameter x2.5,
warm/high-p and cold/low-p. Worst non-sliver rel-err 6e-16 (machine
epsilon); same per-slot sliver rule as G1 (the FMA-contracted
reference remnants). Invariants: per-species cross-mode mass
conservation, strict aitken number loss, gases untouched, and the
pair-less modes (coarse, MAM5's coarse_strat) bit-identical through
the call.
Full suite green (259).
… G4a The exact one-call chain gasaerexch -> newnuc -> coag on grid-cell means, with CAM's del_h2so4_aeruptk bookkeeping: the H2SO4 consumed by condensation is measured as the positive-down difference across the gasaerexch call and handed to nucleation so already-condensed vapour is not double-counted (aero_model.F90:1191-1214, via the sibling repo's mam_coupling_cam transcription). Sub-stepping (A6, owner-approved) deliberately does NOT live in this function: it will wrap the WHOLE per-step physics in the driver (G4b), so n_substeps = n is semantically identical to running the box at deltat/n — the exact quantity the Fortran dt-convergence study varied and found a 2.08x spread over. Validation: tools/capture_microphys (sibling repo) calls the real Fortran chain over 12 cases x both topologies (tropospheric + both stratospheric sulfeq kinds, two gas loadings, production on/off): worst non-sliver rel-err 1.5e-12 — the G1-G3 ulp-level form differences compounded through three chained stages; gated 5e-12 with the per-slot sliver rule. Sulfur (gas + all so4 modes) closes through the chain at 1e-13 per case, and a dedicated test proves the aeruptk bookkeeping is live: zeroing it changes the nucleation answer. Full suite green (264).
MAM5's first physics against an independent reference.
G4b — the assembly:
- calcsize and wateruptake gain optional 'tables' bundles
(CalcsizeTables / WateruptakeTables); None (default) resolves to the
E3SM module constants, bit-identical — the existing suite guards it.
The CAM driver builds bundles from Topology + cam_params with indices
in the gas-window coordinate (calcsize's csizxf lists come from
RENAME's tables in CAM, i.e. the same name-matched aitken->accum walk).
- wateruptake gains qv= (CAM keeps water vapor outside the aerosol
window) and strat= — the wt%-composition solution-volume branch
(wateruptake_sub:583-591). CORRECTS a plan 025 §7 claim: the box
driver sets its tropopause ABOVE the single level under strat
(tropopause_set_box_level(pver+1)), so this branch is live, not dead.
- cam_run_step / cam_run_timesteps: SO2->H2SO4 stub (1e-5/s,
mole-conserving) -> calcsize (fixed per-mode dumfac by default) ->
sulfeq+wtpct+sulden at the pre-wateruptake wet diameter ->
wateruptake -> mmr<->vmr over the window (mechanism adv_mass, number
tracers included) -> the G1-G3 microphysics chain; the substep loop
wraps the WHOLE step so n_substeps = n is running the box at dt/n.
G5 — the findings and the numbers:
- The box reference's lagged-wet-diameter feedback DOES NOT EXIST: the
vendored time-manager shim's is_first_step() is true every step, so
wateruptake re-seeds dgncur_awet = dgncur_a before each sulfeq
computation. Exposed as reseed_dgnwet_each_step (default True = the
reference; False = production CAM's genuine lag). Before this was
found the strat trajectory was 2x off by step 9.
- End-to-end vs the fixdumfac builds, {cam_mam4, cam_mam5} x {trop,
strat}, 120 steps x dt 30 s, all-default namelist: every printed
tracer sits at the reference's own 7-significant-digit print floor
(~5e-7, gated 2e-6), and the one full-precision column — total
sulfur — agrees at 4.5e-15 on all four trajectories.
- A6 substep measurement (vs n=32, dt=30 s): n=1 is 26-78% from
converged across the key tracers, halving per doubling (first-order
splitting; the Fortran study's 2.08x). The shipped default stays
n_substeps = 1: defaults reproduce the reference (the #75-review
convention beats A6's 'default ON' — flagged in the plan as an owner
call). Hosts should pass n_substeps >= 8; a convergence test pins the
direction and rate.
Full suite green (270).
Owner decision 2026-08-26: CAM's un-substepped sequential splitting is 26-78% from the converged answer at dt = 30 s with nucleation active (first-order: halves per doubling), so shipping it as the default answer was judged worse than deviating from the defaults-reproduce-the-reference convention. n = 16 lands at ~1.4-4.4% for 16x cost. - cam_run_step / cam_run_timesteps default n_substeps 1 -> 16, with the measured table and the ADR pointer in the docstring. - ADR-021 records the decision, the measurement, the convention deviation, and the alternatives (1: known-O(50%) answers; 8: still 4-13%; adaptive: the diffrax branch's approach, out of scope). - Reference parity is opt-in and pinned: every parity test passes n_substeps = 1 explicitly; test_documented_defaults locks the default value and reseed_dgnwet_each_step alongside it. Full suite green (270).
FEATURES.md gains a CESM/CAM-variant section (per-component status table with validation numbers and capture-tool pointers); PROGRESS.md records the 2026-08-25/26 arc — sulfeq cluster, G0-G5, ADR-021, the reference findings — noting it lives on feat/cam-driver (PR #74) while merges to main are held.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overnight work toward the CAM driver (plan 024 PR G). 184 tests pass. Every assumption is listed in
docs/plans/025-cam-driver.md§5 for individual accept/reject rather than buried in code.1. The question that had to come first
coupling/amicphys.pyalready has standalone helpers for all four processes, so wiring them into CAM's sequence is tempting — but that would be CAM's sequencing with E3SM's kernels. Settled per process from the discrepancy reports plus one new numerical check:sulfate_equilibgetcoagsbyte-for-byte identical, 1519 linescld = 02. Tropospheric H2SO4 condensation is the same maths
CAM computes
avg_uprt = (1−e^{−KΔt})/Δt, total= gas × avg_uprt, split byfgain ∝ rate. The JAX closed form solves the same competing-sink ODE. Algebraically identical withsrc = 0— checked over 200 random cases, worst relative difference 1.01e-14.3. CAM rename, added as a selectable branch
This was the blocker, and it's smaller than it looked: E3SM's code contains CAM's algorithm as its
optaa /= 40branch, and CAM's path is line-for-line identical to E3SM's legacy rename (27 differing lines of 243, none touching arithmetic). So it's a selector over three decisions, and every quantity it needs was already computed:dryvol_t_del; E3SM skips that and gates after rescalingDefault stays
"e3sm"— asserted, so the production path and its Fortran comparison are bit-unchanged.What validation did and didn't establish
The captured fixture cannot tell the branches apart — at that state the transfer barely engages and both agree exactly. A suite built only on it would conclude the CAM branch was a no-op, so there's a test pinning that.
Oversizing the aitken mode reaches the divergent regime. The discriminating test is CAM's growth gate: with zero growth CAM transfers nothing while E3SM moves 7.69e7 particles — matching the Fortran capture. Also tested: exact conservation both branches, and that CAM's transfer saturates with the growth driver (the tail beyond the boundary is finite — a port scaling without bound would be wrong in a way conservation and positivity can't see).
Not claimed: numeric agreement with the Fortran. That needs the unit/index mapping between CAM's
q/dqdtand the amicphys-local view. A reference set now exists (mam-box-fortran/tools/capture_rename, five growth values); the comparison is the next step and the test module says so.4. A defect found en route
_mam_amicphys_1gridcell's docstring claimed a non-zero cloud fraction "raises a clear error so future workflows don't silently get wrong physics". The code below declined to check, andcldnwas never read at all — so a cloudy cell silently got clear-sky physics applied to the whole cell.The fix had to go outside the jit:
driver.run_stepis@jax.jit, socldnis always a tracer there and a check inside could never fire. The public names are now thin unjitted wrappers that validate first, with a test asserting they are not jitted — if either gets decorated again the guard becomes unreachable.Still to do
Ordered in
docs/plans/025-cam-driver.md§6: numeric rename validation, thesulfate_equilibbranch, then the driver itself and reference comparison at a pinned Fortran tag.A6 needs your sign-off — sub-stepping defaulting ON, diverging from CAM, because reproducing CAM's un-substepped splitting faithfully means reproducing an O(50 %) error at 30 s.
🤖 Generated with Claude Code
https://claude.ai/code/session_01A3zu6nmLHsAUzTkDJFokjv