Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions docs/studio/PROGRESS.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,69 @@ derivations to resolve rather than fixtures.

---

### 2026-08-24 — Results, rebuilt from what d1_globe and inverse_lab do well

Review: "the results after running are shit -- learn from d1_globe and inverse_lab." Studied both
pages (`origin/viz/inverse-lab`, and `plume_dynamics.html` on main, the renamed d1_globe) rather
than guessing. What they have that the results view lacked: **night bands** from the run's own
photolysis, the **sulfur-budget stacked area**, and card polish. Added all three, everything from
the summary the run already produces (no model change).

- **Night bands on every time series**, from the run's OWN J (daylight = any photolysis rate > 0),
aligned to the stored grid by interval index (a time search ties at the edges and can't tell the
interval a step opens from the one it closes -- a test pins it). Absent J draws no bands rather
than a recomputed sun (ADR-005). The OH/HO2 diurnal crash now reads against real day/night.
- **Sulfur budget** as a normalized gas-vs-particle stack (gold SO2, steel particles), the panel
from both reference pages; particle sulfur was already emitted in pptv, so it is a normalization.
- **Particle mass** series (dry H2SO4-equivalent, ug/m3) from particulate_S -- Avogadro and molar
mass only (ASSUMPTION-8), tested against a hand-computed value.

Summary schema 0.1.0 -> 0.2.0 carried the time-resolved spectrum in the prior commit; this adds
`daylight`.

**Fixed a real rendering bug I twice misdiagnosed as a screenshot artifact.** A vertical blue line
ran the full page height. It was not a capture seam: H2SO4 gas starts at exactly 0, and the log
y-scale clamped 0 to Number.MIN_VALUE (~1e-308), giving a finite pixel near -1e308 -- so linePath
drew a segment plunging off the chart, and `overflow: visible` painted it down the page. A zero has
NO position on a log axis; the scale now returns NaN there and the line breaks, exactly as for
missing data. Two scale tests pin it. The lesson recorded: "capture artifact" is a claim to verify
by DOM probe, not a default explanation -- elementsFromPoint on the line is what finally caught it.

375 Python Tier-A (+4), 67 vitest (+2).

---

### 2026-08-21 — The rail moves and the results arrived

Raised in use as three questions — *when does it finish, where do I see results, does the front-end
update?* — and the honest answers were "3–5 minutes", "only in /legacy or the API", and "no". Now:

**Live run states.** The wizard subscribes to the SSE stream the API has pushed since Phase 0; the
rail moves queued → running → succeeded without a reload. The first cut had a lifecycle bug worth
recording: the effect's cleanup closed every socket on any `runs` change — i.e. on the first pushed
update — while the already-watched set prevented reopening, so each stream died the moment it
delivered once and the rail froze at *running* while the server said *succeeded*. Found by driving a
real run and comparing against the API. Sockets now live in a ref, closed only on terminal state or
unmount.

**A results view in the wizard.** Click a run (or submit one — it self-selects): headline tiles
(final SO₂, peak H₂SO₄, peak N, final SA), flags, and three charts from the RunSummary — gas phase
(SO₂/H₂SO₄/OH, pptv), total particle number, and the final size distribution. All read from the
summary, never the npz (ADR-004). The first cut invented its own payload shape and rendered four
confident zeros — headline scalars live on the *run*, the spectrum under `final_size_distribution` —
so the component test's fixture now mirrors the real endpoints, and "no summary yet" states say what
is happening instead of showing an empty shell.

Verified end to end in the browser: a 1-day/40-bin run submitted from the review stage went
running → succeeded at t+18 s with zero reloads, tiles filled (1.7e6 pptv final SO₂, 15.05 pptv peak
H₂SO₄, 3.1e6 cm⁻³ peak N), three charts drew, `open_system_dilution` flagged.

`/legacy` is now fully superseded (task 1.4): the wizard shows results. Deletion is its own small PR.

371 Python Tier-A, 63 vitest (+3).

---

### 2026-08-20 — Interactive dilution, full-profile hover, curated backgrounds, custom modes

Four review requests in one message, all landed (schema 0.4.0 → 0.5.0):
Expand Down
121 changes: 120 additions & 1 deletion studio/modelio/summary.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@

#: Bumped whenever the reduction changes shape or meaning. Stored in every summary so a comparison
#: view can refuse to plot two runs reduced under different rules rather than plotting them anyway.
SUMMARY_SCHEMA_VERSION = "0.1.0"
SUMMARY_SCHEMA_VERSION = "0.2.0"

#: Seconds per day, for the time axis. Named rather than inline (studio/CLAUDE.md).
SECONDS_PER_DAY = 86400.0
Expand Down Expand Up @@ -100,6 +100,31 @@ class SizeDistribution(BaseModel):
total_number_cm3: float


class SizeDistributionHistory(BaseModel):
"""The number spectrum over time: dN/dlogDp on (time x bin), DRY diameters.

Added in summary 0.2.0, prompted by review: the final spectrum alone hides nucleation, because
by the end of a run the burst has grown and coagulated out of the small bins -- the reviewer
read that as "no nucleation" when the npz showed dN/dlogDp peaking at 1.9e7 cm^-3 twelve hours
in. This carries what a banana plot and a time slider need.

Time is decimated by a UNIFORM stride (never a coarsening grid -- CAVEATS.md: non-uniform
resampling aliases the morning number spikes by up to 8x), capped so a 60-day run stays a few
hundred kB of JSON. ``stride`` says what was kept, so nobody mistakes the sampling for the
model's own resolution.
"""

model_config = ConfigDict(extra="forbid", frozen=True)

time_days: tuple[float, ...]
diameter_um: tuple[float, ...]
#: Row i is dN/dlogDp [cm^-3] across the bins at time_days[i].
dn_dlogdp_cm3: tuple[tuple[float, ...], ...]
basis: Basis = Basis.DRY
#: Every ``stride``-th stored step was kept (1 = everything).
stride: int


class ConservationCheck(BaseModel):
"""A budget check, or an explicit statement that it does not apply.

Expand Down Expand Up @@ -135,8 +160,13 @@ class RunSummary(BaseModel):
termination: TerminationReason = TerminationReason.UNKNOWN
flags: tuple[SummaryFlag, ...] = ()
time_days: tuple[float, ...] = ()
#: True where the run's own photolysis was active (daylight), aligned to ``time_days``. Empty
#: when the npz carried no J. The results view shades the complement as night bands -- the
#: same day/night the chemistry actually saw, not a recomputed sun.
daylight: tuple[bool, ...] = ()
series: dict[str, Series] = Field(default_factory=dict)
final_size_distribution: SizeDistribution | None = None
size_distribution_history: SizeDistributionHistory | None = None
sulfur_conservation: ConservationCheck | None = None

def write(self, path: Path) -> Path:
Expand Down Expand Up @@ -172,11 +202,60 @@ def read(cls, path: Path) -> RunSummary:
("V_ratio", "1", Basis.NOT_APPLICABLE, "plume volume expansion V(t)/V0"),
)


def _particle_mass_series(data: dict[str, Any]) -> Series | None:
"""Dry particulate mass as H2SO4-equivalent [ug m^-3], from the stored sulfur count.

``particulate_S`` is molecules of S per cm^3 in the particle phase. Under ASSUMPTION-8 (all
aerosol is pure sulfate) every particulate S atom sits in one H2SO4 unit, so mass follows from
the molar mass and Avogadro alone -- a unit conversion, not new physics, which is why it is
allowed to live here. DRY mass: the water the wet particle carries is deliberately excluded,
and the name says so.

ug/m^3 = molec/cm^3 * (M_H2SO4 / N_A) [g] * 1e6 [ug/g] * 1e6 [cm^3/m^3]
"""
if "particulate_S" not in data:
return None
from studio.science.constants import AVOGADRO, H2SO4_MOLAR_MASS_G_PER_MOL

count = np.asarray(data["particulate_S"], dtype=np.float64)
mass = count * (H2SO4_MOLAR_MASS_G_PER_MOL / AVOGADRO) * 1e12
return Series(
values=tuple(float(v) for v in mass),
unit="ug m^-3",
basis=Basis.DRY,
description=(
"particulate mass as dry H2SO4-equivalent (ASSUMPTION-8: pure sulfate; excludes "
"aerosol water)"
),
)


#: Species whose number density IS its sulfur content -- each carries exactly one S atom. Used for
#: the closed-box budget check. ``particulate_S`` is added separately; it is already a sulfur count.
_SULFUR_GAS_SPECIES = ("SO2", "SO3", "H2SO4")


def _daylight(data: dict[str, Any], time_s: np.ndarray) -> tuple[bool, ...]:
"""Daylight per stored step, from the run's photolysis J (any rate > 0 means the sun was up).

J is stored on interval MIDPOINTS (J_tmid), one fewer than the step edges; each step takes the
daylight flag of the interval it opens. Absent J -> empty, and the caller draws no bands rather
than a guessed sun (ADR-005).
"""
if "J" not in data or "J_tmid" not in data:
return ()
active = np.asarray(data["J"], dtype=np.float64).max(axis=1) > 0.0
if active.size == 0:
return ()
# J has exactly one entry per interval, in order, so interval i's flag is active[i] regardless
# of the midpoint values -- index alignment, not a time search (a search ties at the edges and
# cannot tell the interval a step OPENS from the one it closes). Step i opens interval i; the
# final edge has no interval after it, so it repeats the last flag. The clip also covers a
# mismatched npz (J not exactly one-per-interval) without inventing a sun.
return tuple(bool(active[min(i, active.size - 1)]) for i in range(len(time_s)))


def summarise_state_npz(
path: Path,
*,
Expand Down Expand Up @@ -234,18 +313,58 @@ def summarise_state_npz(
if diluting:
flags.append(SummaryFlag.OPEN_SYSTEM_DILUTION)

mass_series = _particle_mass_series(data)
if mass_series is not None:
series["particle_mass_ug_m3"] = mass_series

# Particulate sulfur as a mixing ratio, converted with the SAME air number density the gas
# series use -- so "H2SO4 in gas and in particles" is one quantity on one axis (pptv), instead
# of pptv and molec cm^-3 sharing a scale, which is the dual-axis lie in disguise. The raw
# molec cm^-3 series stays alongside for anyone comparing against the npz.
if "particulate_S" in data:
particulate = np.asarray(data["particulate_S"], dtype=np.float64)
series["particulate_S_pptv"] = Series(
values=tuple(float(v) for v in particulate / air_number_density * 1e12),
unit="pptv",
basis=Basis.NOT_APPLICABLE,
description="particle-phase sulfur as a mixing ratio (per S atom, ASSUMPTION-8)",
)

return RunSummary(
config_hash=config_hash,
label=label,
termination=termination,
flags=tuple(flags),
time_days=tuple(time_s / SECONDS_PER_DAY),
daylight=_daylight(data, time_s),
series=series,
final_size_distribution=_final_size_distribution(data),
size_distribution_history=_size_distribution_history(data),
sulfur_conservation=_sulfur_budget(data, species, state, diluting=diluting),
)


#: Cap on kept time samples in the history. 240 keeps a 60-day run's spectrum near 350 kB of JSON
#: while a 10-minute-step 1-day run (147 steps) passes through whole.
_HISTORY_MAX_SAMPLES = 240


def _size_distribution_history(data: dict[str, Any]) -> SizeDistributionHistory | None:
"""The (time x bin) spectrum, uniformly strided to at most ``_HISTORY_MAX_SAMPLES`` rows."""
if "dNdlogDp" not in data or "dp_mid_um" not in data or "t" not in data:
return None
spectrum = np.asarray(data["dNdlogDp"], dtype=float)
times = np.asarray(data["t"], dtype=float) / 86400.0
stride = max(1, int(np.ceil(len(times) / _HISTORY_MAX_SAMPLES)))
kept = slice(None, None, stride)
return SizeDistributionHistory(
time_days=tuple(float(v) for v in times[kept]),
diameter_um=tuple(float(v) for v in np.asarray(data["dp_mid_um"], dtype=float)),
dn_dlogdp_cm3=tuple(tuple(float(v) for v in row) for row in spectrum[kept]),
stride=stride,
)


def _final_size_distribution(data: dict[str, Any]) -> SizeDistribution | None:
"""The last spectrum. ``dp_mid_um`` and ``dNdlogDp`` are DRY (see the module docstring)."""
if not {"dp_mid_um", "dNdlogDp", "n_cm3"} <= set(data):
Expand Down
109 changes: 109 additions & 0 deletions studio/tests/unit/test_modelio_summary.py
Original file line number Diff line number Diff line change
Expand Up @@ -234,3 +234,112 @@ def test_summarising_a_real_archived_run(paper_ensemble_runs: Path) -> None:
}
assert summary.sulfur_conservation is not None
assert summary.sulfur_conservation.status == "not_applicable"


@pytest.mark.tier_a
def test_the_history_carries_the_spectrum_uniformly_strided(tmp_path: Path) -> None:
"""Summary 0.2.0: dN/dlogDp over (time x bin), uniform stride, capped sample count.

Uniform, because CAVEATS.md records that non-uniform coarsening aliases the morning number
spikes by up to 8x -- and the whole point of the history is that a reviewer could not see
nucleation in the final spectrum alone.
"""
import math

import numpy as np

from studio.modelio.summary import _HISTORY_MAX_SAMPLES, summarise_state_npz

steps, bins = 1000, 8 # forces decimation: 1000 > _HISTORY_MAX_SAMPLES
times = np.linspace(0.0, 10 * 86400.0, steps)
spectrum = np.tile(np.linspace(1.0, 8.0, bins), (steps, 1)) * (1.0 + times[:, None] / 86400.0)
path = tmp_path / "state.npz"
np.savez(
path,
t=times,
x=np.ones((steps, 2)),
species=np.array(["SO2", "OH"]),
M=1.0e18,
dNdlogDp=spectrum,
dp_mid_um=np.logspace(-3, 1, bins),
)
history = summarise_state_npz(path).size_distribution_history
assert history is not None
assert len(history.time_days) <= _HISTORY_MAX_SAMPLES
assert history.stride == 5 # ceil(1000 / 240)
gaps = {round(b - a, 9) for a, b in zip(history.time_days, history.time_days[1:], strict=False)}
assert len(gaps) == 1, f"stride must be uniform, got gaps {gaps}"
# The kept rows are the original rows, not interpolations.
assert history.dn_dlogdp_cm3[0] == tuple(spectrum[0])
assert history.dn_dlogdp_cm3[1] == tuple(spectrum[5])
assert math.isclose(history.time_days[0], 0.0)


@pytest.mark.tier_a
def test_particle_mass_is_the_unit_conversion_it_claims(tmp_path: Path) -> None:
"""molec S / cm^3 -> ug/m^3 as dry H2SO4 (ASSUMPTION-8), against a hand-computed value."""
import numpy as np

from studio.modelio.summary import summarise_state_npz
from studio.science.constants import AVOGADRO, H2SO4_MOLAR_MASS_G_PER_MOL

count = 6.153e9 # molec/cm^3, arbitrary
path = tmp_path / "state.npz"
np.savez(
path,
t=np.array([0.0, 600.0]),
x=np.ones((2, 1)),
species=np.array(["SO2"]),
M=1.0e18,
particulate_S=np.array([0.0, count]),
)
series = summarise_state_npz(path).series["particle_mass_ug_m3"]
expected = count * (H2SO4_MOLAR_MASS_G_PER_MOL / AVOGADRO) * 1e12
assert series.values[1] == pytest.approx(expected, rel=1e-12)
assert series.unit == "ug m^-3"
assert series.basis.value == "dry"


@pytest.mark.tier_a
def test_daylight_comes_from_the_runs_own_photolysis(tmp_path: Path) -> None:
"""Night bands are the day/night the CHEMISTRY saw (J > 0), not a recomputed sun.

J lives on interval midpoints, one fewer than the step edges; each step takes the flag of the
interval it opens. A run with no J gets an empty tuple, and the results view draws no bands
rather than guessing (ADR-005).
"""
import numpy as np

from studio.modelio.summary import summarise_state_npz

# Six steps over 24 h; J on five midpoints, dark in the middle of the day.
times = np.linspace(0.0, 86400.0, 6)
jmid = (times[:-1] + times[1:]) / 2
jvals = np.array([[1.0], [1.0], [0.0], [0.0], [1.0]]) # night in the two middle intervals
path = tmp_path / "state.npz"
np.savez(
path,
t=times,
x=np.ones((6, 1)),
species=np.array(["SO2"]),
M=1.0e18,
J=jvals,
J_tmid=jmid,
)
daylight = summarise_state_npz(path).daylight
assert len(daylight) == len(times)
assert daylight[0] is True and daylight[1] is True
assert daylight[2] is False and daylight[3] is False
assert daylight[-1] is True


@pytest.mark.tier_a
def test_no_photolysis_means_no_daylight_bands(tmp_path: Path) -> None:
"""Absent J -> empty daylight, so the UI shows no bands rather than a fabricated day/night."""
import numpy as np

from studio.modelio.summary import summarise_state_npz

path = tmp_path / "state.npz"
np.savez(path, t=np.array([0.0, 600.0]), x=np.ones((2, 1)), species=np.array(["SO2"]), M=1e18)
assert summarise_state_npz(path).daylight == ()
Loading
Loading