Skip to content

Add configurable feature-mask tasks for mesh and ocean components - #592

Open
xylar wants to merge 8 commits into
E3SM-Project:mainfrom
xylar:add-region-mask-framework
Open

Add configurable feature-mask tasks for mesh and ocean components#592
xylar wants to merge 8 commits into
E3SM-Project:mainfrom
xylar:add-region-mask-framework

Conversation

@xylar

@xylar xylar commented May 28, 2026

Copy link
Copy Markdown
Collaborator

Summary

This PR adds a shared framework for computing geometric-feature region and transect masks on MPAS meshes, plus configurable mesh and ocean tasks that expose the workflow through Polaris.

Detailed Description

The PR adds a reusable ComputeFeatureMasksStep that builds mask feature collections from geometric_features, validates feature object and mask types, computes MPAS region or transect masks with mpas_tools, and writes both mask NetCDF output and the source GeoJSON. The workflow supports configurable mesh inputs, mesh metadata, mask groups, mask types, multiprocessing settings, subdivision options, and transect edge signs.

The branch also registers configurable feature-mask tasks under both the mesh and ocean components. The ocean task subclasses the shared step so native MPAS-Ocean or Omega mesh files are opened through the ocean I/O layer before using the same mask-generation logic. nRegions and nTransects are added to the MPAS-Ocean-to-Omega dimension map so mask output can be written in native Omega names.

get_feature_mask_steps() is available for workflows that already have an upstream mesh-producing step; it takes the mesh step and filename explicitly rather than special-casing base or culled meshes.

The "MOC Basins" regions get special treatment. A southern-boundary transect is added for each region, and the combined file is given the mocBasinsAndTransects prefix that E3SM and MPAS-Analysis expect.

That MOC behavior lives in polaris/tasks/mesh/spherical/feature_masks/moc.py, not in the ocean step. Both pieces of it — the filename convention and appending the transects — are model-agnostic: they operate on standard MPAS meshes and masks, and neither needs a step instance. Keeping them in a leaf module means a step in another component can subclass the model-neutral ComputeFeatureMasksStep and reuse them without inheriting the Omega name translation, and so without having to supply an [ocean] config section. ComputeOceanFeatureMasksStep delegates to the same two functions, so the MOC logic exists in exactly one place. polaris/tasks/e3sm/init/component_inputs/ is the motivating caller: MPAS-Ocean reads the per-mesh MOC file at run time through its regionalMasksInput and transectMasksInput streams, so it is a component input rather than an analysis product.

Testing

Unit tests cover feature object-type detection, mask-type validation, shared-step configuration, configurable setup, output metadata and filenames, and a full step run against a small synthetic mesh.

The MOC helpers are tested directly, including that the module's own imports name neither polaris.ocean nor polaris.tasks.ocean, and that a step in another component can subclass the model-neutral step, use both helpers, and set up against a config with no [ocean] section. On the ocean side the tests cover the Omega name-translation boundary and the delegation to the shared helpers.

pytest tests/ passes in full (365 tests), and pre-commit run --all-files is clean.

Checklist

  • User's Guide has been updated
  • Developer's Guide has been updated
  • API documentation in the Developer's Guide (api.md) has any new or modified class, method and/or functions listed
  • Documentation has been built locally and changes look as expected
  • Testing comment in the PR documents testing used to verify the changes

@xylar

xylar commented May 28, 2026

Copy link
Copy Markdown
Collaborator Author

The idea is that this shared step will be used in later workflows to generate masks for use in online Omega analysis members (e.g. the MOC and regional stats) as well as offline analysis here in Polaris.

@xylar xylar self-assigned this May 28, 2026
@xylar xylar added ocean Related to the ocean component mesh Related to the mesh component labels May 28, 2026
@xylar

xylar commented May 28, 2026

Copy link
Copy Markdown
Collaborator Author

Testing

On Chrysalis, I set up:

polaris setup -n 184 --model omega -w $POLARIS_SCRATCH/feature-mask

Then, in the compute step directory, I symlinked an Omega initial condition file:

cd /lcrc/group/e3sm/ac.xylar/polaris_1.0/chrysalis/test_20260528/feature-mask/ocean/feature_masks/configurable/compute
ln -s /lcrc/group/e3sm/public_html/polaris/ocean/realistic_global/ocean.QU.240km.151209.omega.teos10eos.nc .

I edited the feature_masks.cfg with the mesh file and name:

# options for creating region or transect masks on an MPAS mesh
[feature_masks]

# Path to an existing standard MPAS mesh file for configurable use
# source: /gpfs/fs1/home/ac.xylar/e3sm_work/polaris/add-region-mask-framework/polaris/tasks/mesh/spherical/feature_masks/feature_masks.cfg
mesh_filename = ocean.QU.240km.151209.omega.teos10eos.nc

# Mesh name used in output filenames and metadata
# source: /gpfs/fs1/home/ac.xylar/e3sm_work/polaris/add-region-mask-framework/polaris/tasks/mesh/spherical/feature_masks/feature_masks.cfg
mesh_name = oQU240

When I ran the step, I got:

$ ncdump -h oQU240_oceanBasins20240830.nc 
netcdf oQU240_oceanBasins20240830 {
dimensions:
	nRegions = 10 ;
	StrLen = 64 ;
	nCells = 7153 ;
	nVertices = 15211 ;
variables:
	char constituents(nRegions, StrLen) ;
	int regionCellMasks(nCells, nRegions) ;
	char regionNames(nRegions, StrLen) ;
	int regionVertexMasks(nVertices, nRegions) ;
	char zmax(nRegions, StrLen) ;
	char zmin(nRegions, StrLen) ;

// global attributes:
		:mesh_name = "oQU240" ;
		:mask_group = "Ocean Basins" ;
		:feature_object_type = "region" ;
		:geometric_features_prefix = "oceanBasins" ;
		:geometric_features_date = "20240830" ;
		:source_mesh_filename = "ocean.QU.240km.151209.omega.teos10eos.nc" ;
		:history = "Thu May 28 14:18:26 2026: ncks -O -5 _tmp_oQU240_oceanBasins20240830.netcdf4.nc oQU240_oceanBasins20240830.nc\n",
			"Thu May 28 14:18:24 2026: /gpfs/fs1/home/ac.xylar/e3sm_work/polaris/add-region-mask-framework/pixi-env/.pixi/envs/default/bin/polaris serial" ;
		:NCO = "netCDF Operators version 5.3.9 (Homepage = http://nco.sf.net, Code = http://github.com/nco/nco, Citation = 10.1016/j.envsoft.2008.03.004)" ;
}

So far, no attempt to translate these variable names into an Omega-friendly format but the hooks are in place to do that later.

@xylar
xylar force-pushed the add-region-mask-framework branch 2 times, most recently from 0c64ad5 to 747dafc Compare May 28, 2026 20:36
Comment on lines +6 to +7
nRegions: NRegions
nTransects: NTransects

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure we actually want these. We may instead want dimensions that are specific to different region groups.

xylar added 5 commits August 12, 2026 06:06
This step computes feature masks for a given region or transect
group for a given MPAS mesh.  A related standalone task allows
a user to point to a mesh, give it a name, and choose a feature
group.
The ocean version can convert mesh variables from Omega format
if an Omega horizontal mesh is provided as an input.  It
also has hooks for translating the mesh variables to Omega's
preferred format (TBD).
The sourthern transects are added for each MOC region, and the
file is writtent with a special prefix to indicate this.
@xylar
xylar force-pushed the add-region-mask-framework branch from ec94852 to ab51378 Compare August 12, 2026 11:10
xylar and others added 3 commits August 12, 2026 07:13
The MOC behavior lived only in `ComputeOceanFeatureMasksStep`, which
bundles it with ocean-model I/O from `OceanIOStep`.  A step in another
component that wants the MOC filename convention and the southern-
boundary transects was forced to inherit the Omega name translation
too, and so to supply an `[ocean]` config section.

Move the two model-agnostic pieces into
`polaris/tasks/mesh/spherical/feature_masks/moc.py`: the
`mocBasinsAndTransects` filename convention and appending the southern-
boundary transects.  Neither needs `self`, and neither touches an ocean
model, so a leaf module holds them rather than a shared base class.

`ComputeOceanFeatureMasksStep` now delegates to them, keeping its
behavior and its API exactly as they are.  Nothing about the produced
file changes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Cover the leaf module directly: the filename convention, appending the
transects, and dropping the string variables that are incompatible with
CDF5.  These assertions move over from the ocean tests, which patched
`add_moc_southern_boundary_transects` in a module that no longer imports
it.

What is left on the ocean side is the delegation boundary: that the MOC
filename matches `moc_masks_filename()` so the two cannot silently
drift, that `_post_process_masks` hands the masks, the mesh, and its
logger to `add_moc_transects()`, and that other mask groups are still a
no-op.

Also check the property that makes the helpers reusable, in the two ways
it can be checked.  A `sys.modules` test is not one of them, since
`polaris/tasks/__init__.py` imports every component: instead, assert
that the module's own imports name neither `polaris.ocean` nor
`polaris.tasks.ocean`, and that a step in another component can subclass
the model-neutral step, use both helpers, and set up against a config
with no `[ocean]` section.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Describe the leaf module where the mesh feature-mask page documents the
rest of the model-neutral implementation, including why importing
`mpas_tools.ocean.moc` does not drag in `polaris.tasks.ocean` and what
that buys a step in another component.

The ocean page keeps its MOC section but now describes the two hooks as
delegating, and drops the claim that the mesh package must not import
`add_moc_southern_boundary_transects`, which is what this refactor
changes.

The user's guide is unchanged: nothing about the produced file moved.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

mesh Related to the mesh component ocean Related to the ocean component

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant