Compute MeshScaling for variable-resolution meshes

[hyungyukang]

This PR adds spatially variable MeshScalingDel2 and MeshScalingDel4 in HorzMesh.cpp to properly account for changes in mesh resolution, following the MPAS-Ocean implementation.

This PR resolves issue #476. As discussed there, UseRefWidth is set to true by default so that MeshScaling is computed from cell width rather than from MeshDensity (Eqs. (1) and (2) of Hoch et al., JAMES 2020).

Checklist

• Documentation:

  • User's Guide has been updated
  • Documentation has been built locally and changes look as expected

• Linting

  • Pre-commit run locally

• Building

  • CMake build does not produce any new warnings from changes in this PR

• Testing

  aurora, oneapi-ifx, mpich

  • CTests Pass
  • Polaris omega_pr Pass

  chrysalis, oneapi-ifx, openmpi

  • CTests Pass
  • Polaris omega_pr Pass

  frontier, craygnu, mpich

  • CTests Pass
  • Polaris omega_pr Pass

  frontier, craygnu-mphipcc, mpich

  • CTests Pass
  • Polaris omega_pr Pass

  pm-cpu, gnu, mpich

  • CTests Pass
  • Polaris omega_pr Pass

  pm-gpu, gnugpu, mpich

  • CTests Pass
  • Polaris omega_pr Pass

• Provide relevant details in a comment to the PR titled Testing with the following:

  • Which machines CTest unit tests
    have been run on and indicate that are all passing.
  • The Polaris omega_pr test suite
    has passed, using the Polaris e3sm_submodules/Omega baseline
  • Document machine(s), compiler(s), and the build path(s) used for -p for both the baseline (Polaris e3sm_submodules/Omega) and the PR build
  • Indicate "All tests passed" or document failing tests

[xylar]

I personally think we should get rid of ScaleWithMesh entirely and not have any support for MeshDensity. The tools for mesh creation now just fill it with all ones. so it is not going to be useful in new meshes. The term ScaleWithMesh is super confusing because it seems like it means UseRefWidth.

[xylar]

I thought this is what we already agreed to in #476 (comment)

[hyungyukang]

@xylar , thanks. Right. I kept that part of the code just in case, mainly for comparison with MPAS-Ocean. Since UseRefWidth: true is the default, that approach will not be used. However, I agree that we should remove it to avoid any confusions and potential issues. Will work on it.

[xylar]

I think we can keep it on a testing branch or something for comparison to MPAS-Ocean but I don't want it to become part of the Omega code unless it's buried enough that no user will try to use it.

[cbegeman]

I'm not sure that this is a good idea because it would be easy for users (or buildnml) to neglect to change this option and thus end up applying a different diffusivity than intended. Could this maybe be computed online (at init) as the median cell width? It's not clear to me that this needs to be user-configurable since the user can just change the diffusivities.

[xylar]

I also agree. The MPAS-Ocean equivalent settings have gotten me in trouble so many times. I like the median resolution idea. I think MPAS-Ocean has a setting where it uses the minimum but that's a noisy value. The median would be a lot more robust -- just requiring a global reduction at init.

[hyungyukang]

@cbegeman and @xylar , thanks for your thoughts on this. I agree. I also had some bad experiences with these settings.

To summarize next steps,

• Removal of MeshScaling setting in omega.yml
• Removal of MeshDensity related MeshScaling computations
• Compute the global median cell width ($\Delta x_{med}$) at initialization
• Compute MeshScalingDel2 and MeshScalingDel4 from $\Delta x_{med}$:

$$ MeshScalingDel2 = \frac{\Delta x}{\Delta x_{med}} $$ $$ MeshScalingDel4 = \left(\frac{\Delta x}{\Delta x_{med}} \right)^3 $$

Do you agree with this plan?

As far as I understand, in MPAS-Ocean, the Del2 and Del4 viscosity coefficients for each horizontal resolution are tuned based on the minimum grid spacing. One concern I have is that $\Delta x_{med}$ on a variable-resolution mesh may be larger than the minimum grid spacing, or its theoretical value. Also, $\Delta x_{med}$ may differ between meshes even if they share the same theoretical minimum grid spacing, for example, IcoswISC30E3r5 and EC30to60E2r2. This suggests that we may need to retune the Del2 and Del4 viscosity coefficients for Omega for each different mesh? Please let me know if I am misunderstanding anything.

[xylar]

One concern I have is that $\Delta x_{med}$ on a variable-resolution mesh may be larger than the minimum grid spacing, or its theoretical value.

@hyungyukang, yes, that would be true by definition, so the coefficient would need to be adjusted accordingly. But the median resolution should be much more robust across different meshes that are generated at ostensibly the same resolution (because of noise in the Jigsaw generation process).

The expectation should not be that the same coefficient is right for IcoswISC30E3r5 and EC30to60E2r2, though they wouldn't differ by too much. I helpful piece of information for debugging would be to have Omega output:

• $\Delta x_{med}$
• the min and max values of the del2 and del4 coefficients.

[xylar]

Do you agree with this plan?

Yes, I agree.