Add a design document for the split-explicit time stepper - #516
Add a design document for the split-explicit time stepper#516hyungyukang wants to merge 9 commits into
Conversation
168eb21 to
550884d
Compare
| $$ | ||
| \tilde h^{*} | ||
| =\tilde h^n+\frac{\Delta t}{2}{\cal T}_{\tilde h}, | ||
| \quad \text{with tracer concentration averaged as in Stage 3}, | ||
| $$ (unsplit-reset-provisional-psi) |
There was a problem hiding this comment.
Is this a break from the MPAS-Ocean algorithm? Does this correspond to the MPAS-Ocean lines https://github.com/E3SM-Project/E3SM/blob/fe750f50e98187afe02103349d8363acd7745a9a/components/mpas-ocean/src/mode_forward/mpas_ocn_time_integration_split_ab2.F#L1898-L1914?
| $$ | |
| \tilde h^{*} | |
| =\tilde h^n+\frac{\Delta t}{2}{\cal T}_{\tilde h}, | |
| \quad \text{with tracer concentration averaged as in Stage 3}, | |
| $$ (unsplit-reset-provisional-psi) | |
| $$ | |
| \tilde h^{*} | |
| =0.5 (\tilde h^n+{\tilde h}^{n+1}), | |
| $$ (unsplit-reset-provisional-psi) |
There was a problem hiding this comment.
@cbegeman , Thanks for taking a look at this document.
It follows the same algorithm used in MPAS-Ocean. In Omega, I update the pseudo thickness using updateThicknessByTend with
I also updated the document. Sections 3.3.4 and 3.3.5 were largely repeating Sections 3.2.5 and 3.2.6, except for the velocity-related parts. So I consolidated those sections and now refer back to Sections 3.2.5 and 3.2.6 instead.
| restart preservation of the three split prognostic fields, and rejection of halo | ||
| widths smaller than three. | ||
|
|
||
| ### 5.2 Polaris tests |
There was a problem hiding this comment.
Is it possible/useful to run a barotropic test case with split-explicit to verify that the baroclinic code isn't affecting the barotropic dynamics?
There was a problem hiding this comment.
That’s a great suggestion. Thanks. Do you have a particular barotropic test case in Polaris that you would recommend? I remember there are several options, but if you have one in mind, I can start with that.
There was a problem hiding this comment.
I think the manufactured_solution case is probably the best place to start
| The split-explicit method involves the following sequence: | ||
|
|
||
| - Decompose velocity into barotropic and baroclinic components. | ||
| - Advance the baroclinic velocities using a large timestep, and compute the vertically averaged forcing, the $\overline{G}$ term. |
There was a problem hiding this comment.
Without an equation the $\overline{G}$ term. doesn't add information. I'd suggest removing it or citing an equation
|
|
||
| ## 1. Overview | ||
|
|
||
| To enhance computational efficiency by allowing longer timesteps, ocean models require split barotropic-baroclinic time stepping methods. The implementation described here is based on the approach of [Higdon (2005)](https://www.sciencedirect.com/science/article/pii/S0021999104005236) and the MPAS-Ocean `split_explicit` scheme, with modifications for Omega's non-Boussinesq pseudo-height $\tilde{z}$ vertical coordinate. |
There was a problem hiding this comment.
would it be useful to reference the v1 equations doc for the pseudo height definition?
|
|
||
| ### 2.1 Requirement: A split time-stepping method in the pseudo-height coordinate | ||
|
|
||
| The algorithm is based on Section 2.3 of [Higdon (2005)](https://www.sciencedirect.com/science/article/pii/S0021999104005236), with modifications to accommodate the $\tilde{z}$-coordinate variables in the non-Boussinesq framework. It accepts the outer time step, requested barotropic time step, barotropic algorithm, number of outer predictor-corrector iterations, number of baroclinic Coriolis iterations, and an option to recompute the velocity split. The `UnsplitRK2` variant advances the full velocity through the baroclinic path and skips all barotropic operations. |
There was a problem hiding this comment.
I'm a little confused by the last sentence. Does this mean in a baroclinic step when splitting it is advancing the full velocity and then corrected? This doesn't make sense to me as it seems that would shrink the max allowed timestep. If this is outside the scope of this doc, feel free to not make any changes, but I'd be curious to learn.
There was a problem hiding this comment.
I think I understand this now. The key is stepping full velocity u_k vs u_k'. The baroclinic code with the full velocity is RK2. My struggle is calling this the 'baroclinic path'. Is there a way to say that for the full velocity the barotropic code is redundant? Maybe the simple change is
| The algorithm is based on Section 2.3 of [Higdon (2005)](https://www.sciencedirect.com/science/article/pii/S0021999104005236), with modifications to accommodate the $\tilde{z}$-coordinate variables in the non-Boussinesq framework. It accepts the outer time step, requested barotropic time step, barotropic algorithm, number of outer predictor-corrector iterations, number of baroclinic Coriolis iterations, and an option to recompute the velocity split. The `UnsplitRK2` variant advances the full velocity through the baroclinic path and skips all barotropic operations. | |
| The algorithm is based on Section 2.3 of [Higdon (2005)](https://www.sciencedirect.com/science/article/pii/S0021999104005236), with modifications to accommodate the $\tilde{z}$-coordinate variables in the non-Boussinesq framework. It accepts the outer time step, requested barotropic time step, barotropic algorithm, number of outer predictor-corrector iterations, number of baroclinic Coriolis iterations, and an option to recompute the velocity split. Since the `UnsplitRK2` variant advances the full velocity the barotropic code is redundant. |
?
|
|
||
| ### 2.2 Requirement: Stable time integration for long-term high-resolution simulations | ||
|
|
||
| Stability constrains the maximum allowable timestep, which in turn affects computational cost. The implemented split-explicit time stepping methods must allow for reasonably long timesteps while preventing numerical instabilities, such as those arising from internal gravity waves and barotropic modes, which are particularly important in global-scale and high-resolution ocean modeling. At a minimum, the time-stepping approach used in Omega V2 should accommodate the same timestep sizes as MPAS-Ocean for both the baroclinic and barotropic subsystems since Omega V2 is non-Boussinesq but hydrostatic. |
There was a problem hiding this comment.
| Stability constrains the maximum allowable timestep, which in turn affects computational cost. The implemented split-explicit time stepping methods must allow for reasonably long timesteps while preventing numerical instabilities, such as those arising from internal gravity waves and barotropic modes, which are particularly important in global-scale and high-resolution ocean modeling. At a minimum, the time-stepping approach used in Omega V2 should accommodate the same timestep sizes as MPAS-Ocean for both the baroclinic and barotropic subsystems since Omega V2 is non-Boussinesq but hydrostatic. | |
| Stability constrains the maximum allowable timestep, which in turn affects computational cost. The implemented split-explicit time stepping methods must allow for reasonably long timesteps while preventing numerical instabilities, such as those arising from internal gravity waves and barotropic modes, which are particularly important in global-scale and high-resolution ocean modeling. At a minimum, the time-stepping approach used in Omega V2 should accommodate the same timestep sizes as MPAS-Ocean for a given resolution for both the baroclinic and barotropic subsystems since Omega V2 is non-Boussinesq but remains hydrostatic. |
|
|
||
| ### 3.1 Barotropic (external) and baroclinic (internal) mode splitting | ||
|
|
||
| The split-explicit method separates ocean velocity into depth-integrated barotropic and depth-dependent baroclinic components. This separation allows computationally expensive baroclinic modes to run at longer timesteps and computationally efficient barotropic modes to run at shorter timesteps, enhancing computational efficiency and accuracy. |
There was a problem hiding this comment.
| The split-explicit method separates ocean velocity into depth-integrated barotropic and depth-dependent baroclinic components. This separation allows computationally expensive baroclinic modes to run at longer timesteps and computationally efficient barotropic modes to run at shorter timesteps, enhancing computational efficiency and accuracy. | |
| The split-explicit method separates ocean velocity into depth-integrated barotropic and depth-dependent baroclinic components. This separation allows computationally expensive slow baroclinic modes to run at longer timesteps and computationally efficient fast barotropic modes to run at shorter timesteps, enhancing computational efficiency and accuracy. |
There was a problem hiding this comment.
also, why does splitting improve accuracy? it seems a consequence of the chosen time stepper, not the splitting itself, but I'm probably missing something
| $$ (split-steric-height-derivation) | ||
|
|
||
|
|
||
| The barotropic continuity and momentum equations are written as follows. |
There was a problem hiding this comment.
Can more detail be provided here? It would make it easier (for at least me to follow). After looking for a bit I can see how these come about
| [\tilde{h}_{i,k}]_e {\bf u}_{e,k}. | ||
| $$ (split-initial-barotropic-velocity) | ||
|
|
||
| For a startup from initial conditions, the code computes this split using the |
There was a problem hiding this comment.
what is this split? Do you mean barotropic velocity?
| \qquad | ||
| \tilde{z}^{*}_{i,k} = \tilde{z}^{n}_{i,k}, | ||
| \quad \text{etc.} | ||
| $$ (split-initial-provisional-variables) |
There was a problem hiding this comment.
what does the * mean here and following?
| $$ (split-stage2-discrete-momentum) | ||
|
|
||
| Throughout this section a hat marks a quantity that lives inside the barotropic | ||
| subcycle and changes from pass to pass, as distinct from the unhatted |
There was a problem hiding this comment.
what is 'pass to pass'?
| whichever subcycle-level pressure anomaly the caller supplies. It is set | ||
| calligraphic rather than hatted so that it cannot be read as any one particular | ||
| buffer, following the same convention as the other calligraphic symbols in this | ||
| document, ${\cal P}_e$, ${\cal R}$, and ${\cal T}$, none of which is a prognostic |
There was a problem hiding this comment.
This paragraph is very confusing. I don't recall calligraphy to this point and there has been a lot to this point, can you explain again what this is? What do you mean by buffer?
| Eq. {eq}`split-stage2-effective-pressure` depends on ${\cal B}'$; the column | ||
| sum is common to all three. | ||
|
|
||
| The column sum $\sum_k[\tilde h_k^*]_e$ uses the flux pseudo thickness on edges |
There was a problem hiding this comment.
do you mean the flux is used to update pseudo thickness which updates the column sum?
| column mass. This is what enforces the [Hallberg and Adcroft (2009)](https://adcroft.github.io/assets/pdf/hallberg_adcroft_OM_2009.pdf) consistency | ||
| described in Section 3.2.4. Note that $B'^{*}$ is unhatted and so is an | ||
| outer-iteration quantity; the hatted $\hat B^{\prime *}$ appearing in the | ||
| predictor-corrector equations below is a different thing, the subcycle predictor |
There was a problem hiding this comment.
do you need to say it's a different thing? can you just say it is the subcycle predictor output?
| output. | ||
|
|
||
| The edge value of the anomaly correction uses the same centered or upwind choice | ||
| selected for pseudo-thickness fluxes. With the upwind choice, ties at zero |
There was a problem hiding this comment.
what does 'ties at zero normal velocity' mean
| [-\nabla \cdot [F_e]^{m+1}]_i. | ||
| $$ (split-stage2-b-corrector) | ||
|
|
||
| On *hatted* subcycle quantities the asterisk denotes a different buffer |
| $$ (split-final-full-velocity) | ||
|
|
||
| $$ | ||
| \psi^{n+1} \quad \text{is retained for pseudo thickness and tracers}, |
There was a problem hiding this comment.
do you mean \phi here? but if so how do tracers (phi) impact pseudo thickness?
| ReinitSplitVelocity: false | ||
| ``` | ||
|
|
||
| #### 4.1.2 Class/structs/data types |
There was a problem hiding this comment.
i'm a little confused by this section. Are these modifications to certain classes from this timestepper? New classes?
| ### 4.2 Methods | ||
|
|
||
| The implemented `doStep` control flow is shown below. Pacer instrumentation and | ||
| the communicator used only by its timing barriers are elided. |
This PR adds the design document for the split-explicit time-stepping method.
The compiled version of the document is available here.
Checklist