Stratified transitions scalar-unroll into per-cell IR → super-linear compile blowup (1.6 GB / 154 s on a 350-state model)

[vsbuffalo]

Summary

Compiling a moderately stratified model produces a very large IR and a long compile,
and the cost is dominated by scalar-unrolling of stratified transitions, not by
integration. A single camdl simulate spends the great majority of its wall-clock in the
compiler before the solver runs.

Observed

A model with dimensions 2 × 5 × 5 over 7 compartments (= 350 states, ~110 source
lines) compiles to a ~1.6 GB IR in ~154 s; the compile log reports a 288,895× source-node
expansion. The RK4 solve itself is fast — essentially all of the ~2.5 min is compilation.

compiled <model>.camdl   1.6GB IR in 154.0s (288895× source)

The model has ~10 transitions, several declared over [c in compartments, …] (e.g. a
death/removal term and an inter-band aging term run over every compartment × every
stratum), plus a few lets with nested sum(...) reductions over two stratum axes.

Likely cause

Transitions declared over [c in compartments, …] combined with the dimension product
appear to be fully scalar-expanded: every (compartment × stratum) cell becomes a flat
scalar IR node with no retained loop/tensor structure, so IR size scales with the full
stratum product rather than with the source. The compartment-indexed transitions are the
worst offenders, since each multiplies the already-large stratum grid by the compartment
count.

Secondary: no cross-invocation compile cache

Repeated CLI invocations of the identical model recompile it from scratch each time. A
parameter sweep run as N separate camdl simulate calls therefore pays N× the full compile
— a 5-point sweep is ~5 × 154 s ≈ 13 min of pure compilation, with zero reuse.

Asks

1. Retain loop/tensor structure in the IR for stratified transitions — emit a vectorized
   kernel over the stratum axes instead of scalar-unrolling into per-cell nodes, so IR size
   scales with the source, not with the stratum product.
2. Cache the compiled artifact keyed on model content across CLI invocations, so a sweep
   (or repeated simulate/fit on an unchanged model) compiles once rather than per call.

Environment

• camdl 0.1.0+0b1fbe12 (2026-07-15)
• macOS / arm64 (Darwin 25.3.0)

[vsbuffalo]

Measured: the stated cause is wrong — 96.4% of the IR is rate_state_grad, not the forward unroll

This issue (and F41, the friction-log entry it was filed from) attributes the 1.6 GB / 154 s
compile to scalar-unrolling of stratified transitions. Measurement says otherwise: the
forward unroll is 0.1% of the IR. The bulk is the unconditionally-emitted state-Jacobian
that gh#275 added.

Evidence

Streaming the serialized field sizes of a real 318 MB compiled fit
(ctl_bb_spray_immladder_anchored, a sibling mean-field-coupled model, 900 transitions):

rate_state_grad serialized: 306.7 MB  (96.4% of IR)
rate_grad serialized:         8.1 MB  ( 2.6%)
rate (forward) serialized:    0.4 MB  ( 0.1%)   <-- what this issue blames
whole IR:                    318.0 MB

And directly on F41's own model (ctl_bb_het, 350 states), with the gh#439 flag:

--no-state-grad:   60.6 MB IR,   4.13 s   (rate_state_grad occurrences: 0)
default (F41):      1.6 GB IR,  154 s

26× smaller, 37× faster. 60.6 MB is 3.8% of 1.6 GB — matching the independently measured
3.6% non-state-Jacobian fraction, confirming ctl_bb_het is ~96% rate_state_grad by the
same mechanism.

What actually happened (timeline verified from git)

commit	date	effect
bf5d13bb	2026-05-31	compact IR serialization — 4.6× faster, 5× smaller. Helped; not the cause.
4664db18	2026-07-07	gh#275 emit rate_state_grad — WrtPop autodiff wired — emits ∂rate/∂compartment unconditionally, for every transition of every model, whether or not it is ever fit by a gradient method. This is the regression.
0c74d51c	2026-07-15	gh#439 camdlc --no-state-grad gates it (manual two-step).
045ecc89	2026-07-16	gh#439 A2 — runtime picks the gate by method, so plain camdl simulate gets it automatically.

Why it's dense on these models: the FOI is mean-field within a village (foi_base[v] reads
inf_vil[v], an aggregate over the whole village block), so every infection transition's
rate depends on every compartment in the block → ∂rate/∂state is a full row per
transition, across the stratum product. That is the ~O(G³) shape gh#439 describes.

So: this is a regression introduced 2026-07-07, nine days before F41 was logged — not a
long-standing property of stratification expansion, and emphatically not the compact-IR work
(which pushed the other way; without it these models would be ~5× worse still).

What this means for this issue

gh#439 already removes the reported symptom for the gradient-free path — which is exactly
F41's use case (a het sweep is simulate; mh/ode fits likewise never read the
state-Jacobian). The vectorized/tensor-IR ask as written would optimize the 0.1%.

Suggest narrowing this issue rather than closing it, since two asks in it remain real and
are untouched by gh#439:

1. No cross-invocation IR cache — an N-point sweep still recompiles the identical model
   per point. Much cheaper now (~4 s vs ~154 s each), but still redundant work.
2. Forward scalar-unroll → vectorized IR — still true as a statement about the compiler,
   but now known to be worth ~0.1% of IR size. Its remaining value is compile time and
   peak RSS on large stratum products, which is a separate (and much smaller) claim than the
   one in the title.

The current title — "Stratified transitions scalar-unroll into per-cell IR → super-linear
compile blowup (1.6 GB / 154 s on a 350-state model)" — attributes the headline numbers to
the wrong mechanism and should be reworded; leaving that call, and the scoping, to @vsbuffalo.

[vsbuffalo]

Verified fixed by gh#439 (0c74d51c + 045ecc89) — and the diagnosis above was wrong; two of the asks should be dropped

Re-measured on 0.1.0+045ecc89 against the same ctl_bb_het model that filed this issue (2 × 5 × 5 × 7 = 350 states).

The blow-up was the state-Jacobian, not scalar-unrolling

camdlc emit	IR	compile	source expansion
full (old default)	1.76 GB	140.0 s	288895×
--no-state-grad (lean)	60.6 MB	4.3 s	9972×

The WrtPop state-Jacobian was 96.6% of the IR. With the runtime now gating it by method, camdl simulate ctl_bb_het goes 154 s → 4.4 s (35×), and the full 5-point sweep ~13 min → ~20 s.

Dynamics-neutral, confirmed on a real stratified model

Re-running het=0.0 on 045ecc89 with the stored run's params/seed reproduces the pre-fix 0b1fbe12 trajectory byte-identical — sha256 ce9c5366…, matching the digest recorded in the old run's run.json. (The SV=2 change re-hashed model identity ctl_bb_het-0ad896eb → -281aebec as expected; param/config/seed hashes unchanged.)

Correcting this issue's asks

• Ask 2 (cache the compiled artifact across CLI invocations) — withdrawn; it already exists. The content-addressed IR cache landed in gh#147 (fbac9d95), long before the 0b1fbe12 binary I filed against. On 045ecc89 all 5 sweep points hit it (cached IR … 2209e5a9, ~3.2 s each). I inferred "recompiles from scratch each point" from ~2 min/point of wall-clock without reading the compile line — that inference was wrong. Whatever the ~2 min actually was (recompile vs. deserializing a 1.76 GB IR), the cause was the IR bloat, and gating removed it.
• Ask 1 (retain loop/tensor structure for stratified transitions) — not needed for this friction. The 288895× expansion was ~97% state-Jacobian; the lean IR compiles in 4.3 s. A vectorized-kernel rewrite may still be warranted for a much larger model, but it should be scoped from fresh evidence rather than the numbers in this issue, which misattribute the cost.

Suggest closing this as resolved by gh#439. Happy to open a narrower issue if the residual 9972× lean expansion ever bites at larger stratification.