Add general realistic-global forward runs by xylar · Pull Request #643 · E3SM-Project/polaris

xylar · 2026-07-05T12:45:33Z

(Full description to come)

This PR extends #612 and generalizes #593 to support both initial conditions from the Polaris workflow and from the Polaris database.

It also provides support for future restart tests and dynamic adjustment workflows.

Checklist

User's Guide has been updated
Developer's Guide has been updated
API documentation in the Developer's Guide (api.md) has any new or modified class, method and/or functions listed
Documentation has been built locally and changes look as expected
Testing comment in the PR documents testing used to verify the changes
New tests have been added to a test suite

xylar · 2026-07-05T12:47:19Z

I have successfully run short (1-day) forward runs for the 240 km unified mesh for both MPAS-Ocean and Omega. Omega requires manually changing the time integrator to RK4.

Nothing in polaris/ocean/coriolis.py is ocean-specific: it imports only polaris.config and polaris.constants, reads mesh coordinates that any MPAS mesh has, and writes fCell/fEdge/fVertex, which are mesh fields rather than ocean fields. Move it to polaris/coriolis.py so sea-ice, land-ice and future components can use it without reaching into the ocean component. The module itself is unchanged; only the 16 imports of it move from polaris.ocean.coriolis to polaris.coriolis.

Move the [coriolis] section from polaris/ocean/ocean.cfg to polaris/default.cfg, so that the options are available to every component rather than only to the ocean. The block moves unchanged, so behaviour is the same: type still defaults to zero. A component with no rotation simply ignores the section, the same way it ignores options it does not read today.

Blank all five [coriolis] options in polaris/default.cfg and raise a ValueError naming whichever option a task failed to set, rather than letting a step silently get no rotation because it forgot to say what it wanted. No task changes: all 16 ocean task families that call add_coriolis_to_dataset already set "type" in their config, and each already sets the parameter belonging to that type -- barotropic_gyre sets beta_plane_f0 at run time and beta_plane_beta in config, and geostrophic sets rotated_sphere_alpha both ways. add_coriolis_to_dataset reads only the parameter belonging to the selected type, so the options left blank for the other types are never touched.

The module has had no direct unit tests: it was exercised only through task init steps. Every function is pure and takes a small dataset, so the tests are cheap. They cover the value each helper produces (2 omega sin(lat) at cells, edges and vertices for the spherical case; alpha=0 matching it and alpha=pi/2 moving the extrema onto the equator for the rotated sphere; uniform fields for constant and zero; linear in y for the beta plane), the attributes the fields carry, dispatch from config to each of the five types, and the error naming whichever option a task left unset.

The module has never appeared in any api.md. Add its six public functions under the framework API reference and a developer-guide page covering both ways to use it: add_coriolis_to_dataset() driven by the [coriolis] config section, and the per-type helpers for steps that should not be configurable. Also drop "ocean" from add_coriolis_to_dataset()'s description of its config argument, which no longer fits now that the module is framework code.

xarray propagates attrs through arithmetic, comparisons, where() and zeros_like(), so a variable computed from another arrives carrying its parent's metadata. Setting individual keys leaves the rest of the parent's metadata behind. set_attrs() replaces a variable's attributes wholesale, which is what it takes to keep a derived field from inheriting a label that no longer describes it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

minLevelCell, maxLevelCell and cellMask were derived from BottomPressure via zeros_like(), comparisons, where() and np.maximum, every one of which propagates attrs, so all three claimed to be a seafloor pressure in Pascals. RefPseudoThickness and vertCoordMovementWeights had no attributes at all. Give each field it creates its own long_name and, where meaningful, units. PseudoThickness, ZTildeMid and ZTildeInterface are labelled here too, so that init_pstar_vertical_coord() produces a fully described coordinate on its own rather than relying on PStarInitStep to finish the job. BottomPressure is the caller's variable, so capture its attributes and put them back after the post-snap overwrite instead of inventing new ones. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Setting long_name and units individually leaves any other attributes the variable inherited from its parent in place, which is how BottomPressure ended up saying both units = Pa and unit = meters. Use set_attrs() so each written field carries exactly the metadata it is given. Drop the re-labelling of PseudoThickness, ZTildeMid and ZTildeInterface, now done by init_pstar_vertical_coord(), and the vertCoordMovementWeights fallback that could never run: init_pstar_vertical_coord() always puts that variable in the dataset first, so the "not in ds" guard was never true. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The globalStats analysis member and its stream both followed output_interval, which ties a handful of scalars to the cadence of the 3-D output. That is the wrong coupling: a run whose output_interval is long enough to keep the 3-D files manageable gets a statistics time series too coarse to see anything in, and a run shorter than its own output interval writes only the record from config_write_output_on_startup -- so its "final" statistics are its initial ones. Add a stats_interval option, defaulting to one day, that drives the globalStatsOutput stream and through it the analysis member's compute interval. The 3-D output_interval is untouched, so the volume of 3-D output does not change. MPAS-Ocean only: Omega's GlobalStats reduction period is baked into the variable names mpaso_to_omega maps ("..._TimeMean1Day"), so its statistics stay daily whatever this is set to, and the config comment says so. Carried up from the dynamic-adjustment branch, where the coarse cadence made the per-stage diagnostics report each stage's starting state as its end state. The schedule that sets a finer interval stays there. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

MPAS-Ocean's two horizontal-mixing scaling flags are nested rather than independent (ocn_meshScaling in mpas_ocn_mesh.F): if (config_hmix_scaleWithMesh) then if (config_hmix_use_ref_cell_width) then ! by cell width else ! by meshDensity else ! no scaling at all 'ref_cell_width' set use_ref_cell_width without scaleWithMesh, so it fell into the outer else and applied no scaling at all. The meshDensity branch does not work either: meshDensity is a legacy field holding (cellWidthMin/cellWidth)**4, and E3SM v4 meshes -- including every mesh Polaris builds -- write it as uniformly 1.0. So every realistic_global mesh has been running unscaled, which went unnoticed because the quasi-uniform meshes reference their own resolution, where the scaling would be about 1 regardless. So 'ref_cell_width' now sets both flags, and it is the only scaling Polaris offers; the meshDensity branch is documented as a thing not to reach for rather than exposed. That leaves the two variable-resolution meshes, where the difference is real, needing a reference width. E3SM's per-grid coefficients are referenced to the finest cells of their own mesh, so u.oi6to18.lr6to10 takes RRSwISC6to18E3r5's mom_del4 = 3.2e09 against a 6 km reference and u.oi.so12to30.lr10 takes SOwISC12to30E3r3's 1.18e10 against 12 km. Both were checked against the meshDensity scaling of the E3SM mesh they borrow from, reconstructed from the mesh file itself: cubic scaling from those references reproduces E3SM's viscosity as a function of cell width to within about 15% across the bulk of each mesh. Also adds mom_del4_div_factor, which scales the divergence part of the biharmonic operator alone. E3SM sets it to 10 on RRSwISC6to18E3r5 and on no other mesh; u.oi6to18.lr6to10 now matches. At 6 km the rotational part is resolved eddies worth keeping sharp while the divergent part is where C-grid grid-scale noise lives, so damping them equally is not what an eddying mesh wants. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The forward package landed on this branch without an api.md section, so none of what it adds -- the task, the Forward step, ForwardStage and the InitialCondition classes -- has a rendered API page. ForwardStage in particular is the piece downstream branches build on: dynamic adjustment constructs its stages by field name, and with no generated page there is nowhere for that documentation to point. Follows the realistic_global.init section's submodule-prefixed style, and sits alphabetically between forcing.jra55 and hydrography.woa23. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The mapping in mpaso_to_omega.yaml sends MPAS-Ocean's global-statistics variables to plain Omega names such as Temperature_SpatialMax, but forward.yaml asked Omega's GlobalStats group for a daily temporal reduction. Omega names a reduction chain "<stem>_TimeMean<period>" and an instantaneous chain plain "<stem>" (AnalysisGroup::buildTemporalChains), so none of the mapped variables would have existed in the output. Configure snapshots instead of reductions, which is what the mapping already assumes. This also drops a constraint the reduction path carried: Omega aborts unless the RestartWrite interval is a whole multiple of an averaging period, and that check applies only to reductions, so restart_interval no longer has to be a whole number of days. With the period no longer baked into the variable names, there is no reason to hard-code it either, so SnapshotPeriod is templated from stats_interval and the option now drives both models rather than MPAS-Ocean alone. duration_to_omega_period converts the MPAS duration string to the form Omega wants, choosing the largest unit that divides the interval exactly. That string is also what Omega echoes into the name of the file it writes (<prefix>_<period>Instants), so it has to be derived once and reused. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

OceanModelStep grew a `target_location` parameter so that a step reading a cached initial condition could pull `graph.info` from the input-file database rather than an upstream step's work directory. That is a second answer to a question the ocean framework already answers elsewhere -- where do the model input files come from -- and it is the answer that collides with the InitialCondition abstraction the realistic_global forward runs use. Drop it. A step whose graph does not come from an upstream work directory now leaves `graph_target` unset and adds `graph.info` itself, which the analysis_members forward step does in its own `setup()` once the model, and so the database subdirectory, is known. That makes a `graph_target` of None meaningful, where before it was neither honored nor rejected: `__init__` set `self.make_graph = True` for it, but `ModelStep.__init__` ran afterwards and set `make_graph` straight back to the argument's default of False, so the graph was never built and MPAS-Ocean got an input file entry pointing at nothing. The dead assignment is gone and the docstring now says what the parameter actually does. DatabaseInitialCondition documented the same false promise -- that the base class would build its graph -- so its docstring is corrected too. Giving it a real graph from the database comes with the rest of the staged-file layout, in a follow-up. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

DatabaseInitialCondition was written before any initial conditions had been uploaded, so it guessed at a name -- ocean.{mesh}.{id}.{eos}.nc under realistic_global/{model} -- and was left unregistered until real files settled the question. They are on the server now, staged by ocn/add-global-ocean-forward under realistic_global/{model}/{mesh_name}, and the two models get different files: mpas-ocean: ocean.{mesh}.{mpaso_id}.zerovel.nc + graph.info.{mpaso_id} omega: ocean.{mesh}.{mpaso_id}.{eos_type}.{omega_id}.nc Follow that, asymmetry and all. It is not a design, it is what is on the server, and renaming means re-staging every file; that waits until realistic_global/init can produce database initial conditions itself. An Omega run with no omega_id now fails with a message naming the missing argument rather than a download error for a wrong filename. One database file supplies everything the model reads, so it is registered under each local name it stands in for: mesh and initial state for both models, and the vertical coordinate as well for Omega. That includes the wind stress, which is why the forcing filename is now asked of the initial condition rather than read straight from [ocean_staged_files]: a step-backed source names its own staged forcing file, and a database source names the initial condition. The forward step no longer has to know which case it is in, and "every realistic_global forward run is wind-forced" stays true for both. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The task built its own StepInitialCondition, so it could only ever run from the shared realistic_global/init chain. That chain has to remap WOA23 and cull topography before the model takes a single step, which makes the task far too expensive to put in a PR suite -- the one thing a cached initial condition is good for. Take the initial condition as an argument instead, and let the caller pass the upstream steps it depends on. A source with no upstream steps passes none, and the task is then just the forward run. The per-mesh factory keeps registering exactly what it did before; it now builds the StepInitialCondition itself, which is also where the mesh's min_res and cell-count estimate belong, since they describe the source rather than the task. subdir_name keeps two tasks on the same mesh apart when they differ only in where their initial condition comes from. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Both steps plot a forward run's output, so they belong beside the step that produces it rather than in the task package that happened to be their first consumer. Every RealisticGlobalForward now gets them, not run by default, which matters most for the long init-chain runs: a statistics time series is the cheapest way to see a run going wrong before it NaNs. StatsAnalysis looked for Omega's statistics in global_stats_1DayTimeStats. That is the name of a *temporal reduction* stream, and nothing has ever configured one, so on Omega the step read a file that was never written. It went unnoticed because the step does not run by default. Omega builds the name from the configured prefix, the analysis period and the kind of output -- "<prefix>_<period>Instants" for instantaneous samples and "<prefix>_<period>TimeStats" for reductions (AnalysisGroup::createAnalysisGroupStreams) -- so the name is now built from the same ForwardStage the forward step renders its config from, and follows stats_interval with it. Viz read the wind stress from init.nc, which is only where it lives when the initial condition came from the database; the init workflow writes a forcing file of its own. It now asks the forward run's initial condition which file holds the forcing, the same question the forcing streams ask. Its mesh and initial-state inputs also go through the [ocean_staged_files] names rather than hard-coded ones, which means adding them in setup() once the model is known. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Every forward task so far has to remap WOA23 and cull topography before the model takes a step, which rules the whole family out of a PR suite. Register the same task against the cached initial conditions that ocn/add-global-ocean-forward staged, on QU.240km and EC30to60E2r2, under cached_forward so a mesh could carry both without a collision. These tasks exist to compare MPAS-Ocean against Omega on one mesh and one initial condition, which is a different purpose from the init-chain tasks: those test the unified meshes themselves in E3SM, where E3SM-like physics is the point. So the per-mesh configs here carry RK4 for MPAS-Ocean rather than the split_explicit_ab2 default, putting both models on the integrator Omega has, along with the time steps and run durations that branch tuned for it. A 10-minute baroclinic step under split time stepping would be untested here in any case. Following that purpose through to the physics needs config options that do not exist yet, and comes next. Neither mesh is built by Polaris, so neither config has a [vertical_grid] section; the vertical grid arrives with the cached initial condition. The test that config files are named for real meshes now counts these as real meshes too. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The KPP boundary layer, the submesoscale parameterization and the pressure-gradient formulation were fixed in the mpas-ocean section of forward.yaml, so nothing could turn them off. That was fine while every forward run was E3SM-like, and it is not now: the same step also serves runs whose purpose is to compare MPAS-Ocean against Omega, where physics Omega has no counterpart for is a defect rather than a feature. Move all three to [realistic_global_forward] alongside use_GM and use_Redi, which were already options for the same reason. The defaults are unchanged E3SM-like values, so every task that exists today runs exactly as it did. use_KPP and use_submesoscale are stated either way rather than only when on, so that turning them off in a config undoes a default that turned them on -- the same reasoning as hmix_scaling. pressure_gradient_type is the exception: blank leaves the MPAS-Ocean default of pressure_and_zmid, since there is no reason to restate a Registry default the model already holds. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

These tasks exist to compare MPAS-Ocean with Omega on one mesh and one initial condition. Gent-McWilliams, Redi, the KPP boundary layer, the submesoscale parameterization and the Jacobian_from_TS pressure gradient have no Omega counterpart, so running them makes the two models less comparable, which is the opposite of what the task is for. Turn them off, leaving MPAS-Ocean on pressure_and_zmid, the ordinary counterpart to Omega's Centered. The init-chain tasks on the unified meshes keep all of it. Their purpose is the other one: testing those meshes in E3SM with MPAS-Ocean, where E3SM's physics is the point and Omega is a smoke test that cannot go further until it has both the physics and an integrator that makes dynamic adjustment practical. The list in these two config files describes what Omega lacks today rather than a considered choice of physics, so expect it to shrink. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…shes The three 240 km mesh configs were ported from Compass's qu240 mesh, which turned on the Leith closure and frazil ice formation. Both were a smoke-test convenience there. That purpose does not apply in Polaris, at least not yet, so neither should be inherited just because it came along with the rest of the port. State them off rather than deleting the lines: the default is already False, but these are the meshes where they were on, and an explicit value records that the departure from Compass is deliberate rather than an oversight. With Leith off, the constant mom_del2 is the harmonic viscosity. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Everything analysis_members did is now done by the cached_forward tasks on the same two meshes and the same cached initial conditions: a short forward run with the global statistics enabled, plus the statistics and map plots, which moved into the forward package earlier in this series. What is left here is the old Forward step and the task that wired it up, both of which duplicate the reusable versions. The omega_pr entry goes with it rather than moving to the new path. The cached task is not the one main ran -- Omega now writes instantaneous statistics rather than the reduction its variable names never matched, and the physics that has no Omega counterpart is stated off rather than left to defaults -- so restoring the entry now would be asserting a result nobody has seen. It comes back once the task has been run under Omega. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The cached tasks have no test that goes all the way from per-mesh config to rendered yaml. The pieces are each covered -- the stage mapping, the initial condition's input files, which options are on -- but nothing checks that they compose into the configuration the model would actually be handed. Render it for both models on both cached meshes: the tuned time step and RK4 reach the replacements, Omega asks for instantaneous statistics on the configured period, and the physics with no Omega counterpart is off and is not quietly put back by forward.yaml. This is as close to a `polaris setup` smoke check as a unit test can get; a real setup would first have to download the cached initial condition. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The user guide described the forward runs as smoke tests and said a careful MPAS-Ocean/Omega comparison "would need its own config options chosen for that purpose". That comparison now exists, as cached_forward, and the config options are chosen for it -- but nothing said so, and nothing explained why the same step is configured two different ways. Replace the analysis_members section with one that covers both tasks together and states the split: cached_forward compares the two models, so it runs only the physics Omega also has and puts both on RK4; forward tests a mesh in E3SM with MPAS-Ocean, so it runs E3SM's physics and split-explicit time stepping, with Omega as a smoke test that cannot go further until it has both the physics and a practical integrator for dynamic adjustment. Say plainly that cached_forward is expected to be temporary. It exists because Omega lacks physics and the Polaris initialization workflow has not been tested end to end; when neither is true, the comparison belongs on meshes Polaris builds itself and this task can go. Its list of disabled physics records what Omega lacks today rather than a considered choice, and should shrink. The physics, forcing and time-integrator sections are reconciled with that, and the mesh, vertical grid, initial condition and cores subsections the task template asks for are filled in for both tasks. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Setting up a cached_forward task crashed with "No option 'eos_type' in section: 'ocean'". The Omega database filename carries the equation of state, and DatabaseInitialCondition reads it from config when it is not given explicitly, but nothing in the forward task's config provided it. It used to come from analysis_members.cfg. The init-chain tasks get it from realistic_global_init.cfg, which the init steps carry -- so the forward step happened to see it there, and only the cached tasks, which set no init workflow up at all, were left without. State it in realistic_global_forward.cfg, so a forward task is self-contained either way. The whole realistic_global family is TEOS-10, and both config files now say so. The unit tests missed this because their fake step supplied eos_type itself, which cannot tell whether the task's own config file provides it. Add a test that builds the config the way the task does and lets the initial condition read from it; with the option removed again it reproduces the traceback above. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The Omega Forcing stream was copied from Omega's own Default.yml, FreqUnits: Never and all, on the assumption that the value was inert because Forcing::init() reads the stream explicitly rather than on a schedule. It is not inert. IOStream::create returns early for 'never' and never registers the stream, warning as it goes; Forcing::readStreamIntoArrays then fails to read it, logs at info level that it is "using zero forcing", and the run continues unforced. So a QU.240km cached run completed, wrote plausible output, and had no wind stress in it. Kinetic energy after one day came out about 30% below the same run with forcing. That is the failure this task is built to make impossible: wind forcing is unconditional here and there is no config option to turn it off. Use OnStartup, which registers the stream and reads it before the first step -- the same value ocn/add-global-ocean-forward arrived at, and a run of that branch logged "Successfully read stream Forcing from file init.nc" where this one did not. Contents stays as the Forcing field group, which is where SfcStressForcingVars::registerFields puts SfcStressZonal and SfcStressMeridional, so it names the same two fields the explicit list would. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The Omega history stream carried State, Tracers and KineticEnergyCell, which on QU.240km is six variables: enough to see that a run went wrong, not enough to see why. The branch this reconciles with wrote 24, and losing 18 of them was an oversight rather than a decision -- it was noted during the audit and then never acted on. Add back AuxiliaryState, SshCell and Eos, which is what those 18 variables are: the vorticity, divergence and del2 terms that show where a problem is, the free surface, and the specific volume and buoyancy frequency that separate a stratification problem from a dynamics one. PseudoThickness is not named, unlike on that branch, because State already provides it; KineticEnergyCell is named even though AuxiliaryState provides it, since it is the field most often wanted. The surface stress is deliberately not added to the restart stream. A continued run re-reads it from the Forcing stream, so carrying it in the restart would only be a second copy that could disagree. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Omega's history stream carries SshCell and the equation-of-state fields while MPAS-Ocean's output had no counterpart to either, so the model comparison these tasks exist for could not be made on the free surface or on density at all. ssh is already mapped to SshCell in mpaso_to_omega.yaml, so open_model_dataset presents both models' free surface under one name. density has no Omega counterpart in the map, since Omega carries the specific volume instead, but it is the closest MPAS-Ocean field to what the Eos group writes and costs little to keep. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Restores the coverage that removing analysis_members gave up, on the task that replaced it, and adds the same coverage for MPAS-Ocean, which never had it. Both models have to keep working here: these tasks exist to be compared against each other, so a regression in either one is a regression in the comparison. Verified on chrysalis before landing. QU.240km/cached_forward passes under Omega (1:18) and MPAS-Ocean (0:54), starting from the same zero-velocity initial condition, and the two agree to 0.04 degC in mean temperature after five days. EC30to60E2r2 passes under Omega (8:18) but stays out of the PR suites at 236853 cells. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

A suite file is a list of path strings with nothing validating them, so a task that is renamed leaves the suite pointing at something gone, and a new entry with a typo covers nothing. Both fail only when someone runs the suite. Adding the cached forward task to the PR suites hit the second case: the first attempt omitted the trailing "/task" that a task subdirectory ends in. Check every suite file in the repo, and while here that no suite lists a task twice. 'cached:' lines are skipped, since they name steps of the task above them rather than a task. That turned up two suites already broken on main: cull_topo_simple.txt and remap_topo_simple.txt name Icos480km and QU240km where the tasks have been icos480km and qu480km for some time. All 28 entries across the two files were unresolvable, so neither suite could have run. Fixed by matching case-insensitively against the registered task paths, which found exactly one match for each. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The developer guide covered the init, woa23 and jra55 tasks and the two diagnostic steps, but said nothing about the package the forward runs are actually built from -- so the Forward step, ForwardStage and InitialCondition were documented only by their docstrings and the API listing. Add a section for it, organized around the split the package is arranged by: initial_condition.py answers where the model input files come from, stage.py answers what settings a run uses, and forward.py runs the model. That split is the reason one step can serve both an init-chain task and a cached one, so it is the thing a reader needs first. The viz and global_stats sections move under it, since both are forward steps rather than free-floating framework. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Both models take the wind stress at cell centres and project it onto edges themselves, and they do not do it the same way by default: MPAS-Ocean uses the two cells adjacent to an edge, Omega uses four. Nothing here said which one to use, so MPAS-Ocean was using its Registry default of two. Set config_bulk_wind_stress_interp_isotropic for every realistic global forward run. Four cells is what E3SM runs with today, which is what the unified-mesh tasks are meant to reproduce. It is also Omega's only option, so the cached tasks get the same momentum on the same edges in both models without anything further being said -- the comparison benefits, but it is not the reason. That distinction matters for whoever revisits this: the isotropic form is slightly less accurate and Omega may not keep it. If it goes, this option should follow E3SM rather than follow Omega. Stated in the mpas-ocean section because there is no Omega option to translate it to; the neutral section is checked by a test that every option in it maps. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Omega bakes the analysis period into the name of the file it writes, so what a stage's global statistics are called depends on the model and on stats_interval. StatsAnalysis worked that out for itself in setup(). That was fine while it was the only reader, and it no longer is: the dynamic-adjustment diagnostics read the same file per stage, and they still look for the pre-snapshot name. A second copy of this rule is how that stays broken -- a reader that guesses the name wrongly does not fail, it finds nothing and quietly falls back to output.nc. Put the rule on ForwardStage, next to stats_period() that it is built from, and have StatsAnalysis ask for it. The stage is the right owner because the period is a property of the stage, and because someone asking what files a stage produces should not have to read a plotting step to find out. The MPAS-Ocean branch comes along for the ride: it returns the name unchanged, which is worth stating rather than leaving implicit, since that name not moving is exactly why the Omega one drifting is easy to miss. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Both new tests check stats_filename() against forward.yaml rather than against string literals. The failure mode being guarded is that the yaml changes -- as it did when Omega's GlobalStats went from daily reductions to snapshots -- and a reader somewhere goes on asking for the old name. A test that hard-codes the expected name changes in lockstep with the reader and notices nothing. The second test pins the part that surprised us: MPAS-Ocean's name is fixed while Omega's moves with stats_interval, so an hourly stage writes its statistics somewhere a daily one does not. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Both models' names now come from ForwardStage.stats_filename(), so the developer guide says so and says why it lives on the stage, rather than describing it as something the plotting step works out. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

density is in the 3-D output stream so that MPAS-Ocean and Omega can be compared on the same fields. It is also the most expensive thing in that stream: it is 3-D, and it is about 13% of the output volume -- 2.4 GB per write on u.oi6to18.lr6to10. A one-day smoke test does not notice; a staged spin-up that writes the stream once per stage does, and the dynamic-adjustment workflow is about to turn it off. Make it a config option rather than deleting it, because which answer is right depends on what the run is for -- the same split that put the boundary layer and the pressure gradient in config. Default on, so the comparison the field exists for keeps working without anything being said. ssh stays unconditional beside it: it is 2-D, costs almost nothing, and is the surface field that makes the two models comparable at all. The replacement is a real bool rather than the 'true'/'false' strings the rest of model_replacements uses, because a jinja {% if %} tests this one instead of substituting it, and jinja counts the non-empty string 'false' as true. A string here would have written density always, and quietly. That widens the return annotation to Dict[str, Any], which the docstring now explains. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Two tests: that the switch adds and removes density while leaving ssh and the rest of the stream alone, and that model_replacements hands jinja a real bool. The second looks pedantic and is the one worth having. Every other entry in that dict is a string, so 'false' is the natural thing to write, and jinja would read it as true -- the switch would appear to work, the config would read correctly, and density would be written anyway. _forward_config lists every option in the section, so it gains the new one. Without that, adding a required option fails 30 unrelated tests, which is how this was found. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The user guide embeds the forward config verbatim, so it carries the new option and the reasoning for its default. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

xylar force-pushed the add-realistic-global-ocean-general-forward branch 2 times, most recently from 1fe3f28 to 64a105c Compare July 5, 2026 15:46

xylar mentioned this pull request Jul 5, 2026

Add dynamic adjustment to realistic, global ocean workflow #644

Draft

6 tasks

xylar force-pushed the add-realistic-global-ocean-general-forward branch 5 times, most recently from 4493593 to 3c30fa5 Compare July 21, 2026 11:20

xylar mentioned this pull request Jul 25, 2026

Fix u.oi30.lr10 culling #672

Merged

4 tasks

xylar force-pushed the add-realistic-global-ocean-general-forward branch from 78299fd to 5363c2c Compare July 27, 2026 11:58

xylar mentioned this pull request Jul 28, 2026

Fix spurious sea-surface height from p-star partial-cell snapping #674

Merged

1 task

xylar force-pushed the add-realistic-global-ocean-general-forward branch 3 times, most recently from 98d8e95 to 1d5245a Compare August 4, 2026 09:16

xylar force-pushed the add-realistic-global-ocean-general-forward branch 4 times, most recently from 433bf72 to 57bcd6b Compare August 6, 2026 11:52

xylar added enhancement New feature or request ocean Related to the ocean component labels Aug 6, 2026

xylar and others added 9 commits August 8, 2026 06:40

Merge branch 'move-coriolis-to-framework' into unified-mesh-base-branch

bef5447

xylar and others added 29 commits August 12, 2026 07:56

Document the density output switch

bc7db0f

The user guide embeds the forward config verbatim, so it carries the new option and the reasoning for its default. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

xylar force-pushed the add-realistic-global-ocean-general-forward branch from 03a38ac to bc7db0f Compare August 12, 2026 13:07

Navigation Menu

Uh oh!

Add general realistic-global forward runs - #643

Add general realistic-global forward runs#643
xylar wants to merge 209 commits into
E3SM-Project:mainfrom
xylar:add-realistic-global-ocean-general-forward

xylar commented Jul 5, 2026

Uh oh!

xylar commented Jul 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

xylar commented Jul 5, 2026

Uh oh!

xylar commented Jul 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant