Skip to content

Merge E3SM master into Omega develop - #521

Merged
sbrus89 merged 125 commits into
developfrom
master
Aug 25, 2026
Merged

Merge E3SM master into Omega develop#521
sbrus89 merged 125 commits into
developfrom
master

Conversation

@sbrus89

@sbrus89 sbrus89 commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

This PR brings in changes from:

Which are needed for the C++ 20 updates in:

Checklist

  • Testing

    aurora, oneapi-ifx, mpich

    • CTests Pass
    • Polaris omega_pr Pass
    • e3sm_omega_developer Pass

    chrysalis, oneapi-ifx, openmpi

    • CTests Pass
    • Polaris omega_pr Pass
    • e3sm_omega_developer Pass

    frontier, craygnu, mpich

    • CTests Pass
    • Polaris omega_pr Pass
    • e3sm_omega_developer Pass

    frontier, craygnu-mphipcc, mpich

    • CTests Pass
    • Polaris omega_pr Pass
    • e3sm_omega_developer Pass

    pm-cpu, gnu, mpich

    • CTests Pass
    • Polaris omega_pr Pass
    • e3sm_omega_developer Pass

    pm-gpu, gnugpu, mpich

    • CTests Pass
    • Polaris omega_pr Pass
    • e3sm_omega_developer Pass
  • Provide relevant details in a comment to the PR titled Testing with the following:

    • Which machines CTest unit tests
      have been run on and indicate that are all passing.
    • The Polaris omega_pr test suite
      has passed, using the Polaris e3sm_submodules/Omega baseline
    • Document machine(s), compiler(s), and the build path(s) used for -p for both the baseline (Polaris e3sm_submodules/Omega) and the PR build
    • Indicate "All tests passed" or document failing tests
    • Document testing used to verify the changes including any tests that are added/modified/impacted.

jonbob and others added 30 commits May 20, 2026 13:14
With cflx_cpl_opt==2, cflx_tend() was called in tphysbc (cam_run1),
which is invoked multiple times during initialization, causing CO2
surface fluxes (CO2_OCN, CO2_FFF, CO2_LND, CO2) to be applied
redundantly and adding duplicate mass to the atmosphere.

Fix (following @huiwanpnnl): split cflx_tend into two paths:
- tphysbc (cam_run1): apply aerosol fluxes only (skip_co2=.true.)
- tphysac (cam_run2): apply CO2 fluxes only (co2_only=.true.)

This preserves the aerosol numerical coupling improvement of
cflx_cpl_opt==2 while moving CO2 tracers back to cam_run2 to
avoid the redundant flux application during the init sequence.

Changes:
- cflx.F90: add optional skip_co2 and co2_only arguments to
  cflx_tend(); build CO2 mask from c_i array to identify CO2
  tracers (CO2_OCN, CO2_FFF, CO2_LND, CO2)
- physpkg.F90: call cflx_tend with skip_co2=.true. in tphysbc;
  add elseif branch in tphysac to call cflx_tend with co2_only=.true.
  when cflx_cpl_opt==2; make get_carbon_sfc_fluxes unconditional
  in tphysac
Add infrastructure for water-tracer and isotope tracking:
- Register WaterTracers and WaterIsotopes process types
- Add CMake build rules and process factory integration
- Create stub process interfaces with identity tendencies
- Add namelist defaults for tracer_count configuration

Part of water isotope infrastructure campaign (spec 001).
In components/eam/src/ and components/homme/src/, the command
  grep -ir 'hyai(1)' *
reveals multiple places where the calculation of air pressure
assumes hybi(1) = 0. This commit adds a few lines to abort
a simulation when hybi(1) is non-zero.
If dosw or dolw is .false. at the first timestep,
set the corresponding heating rates and TOA/SFC fluxes to zero
to avoid floating-point exception in debug runs with LW or SW turned off.
...if all pmid values are higher than a specified threshold (currently set to 450 hPa)
Bugfixes due to separating CO2 tracers.
Move water tracer and isotope processes from physics/water_tracers/
to physics/aux_tracers/{water_tracers,water_isotopes}/ with separate
libraries per process. Water isotopes depend on and extend water tracers.

- Add aux_tracers/CMakeLists.txt with EAMXX_ENABLE_WATER_TRACERS and
  EAMXX_ENABLE_WATER_ISOTOPES options (both default OFF)
- Enforce dependency: isotopes require tracers
- Create separate water_tracers and water_isotopes libraries
- Update register_physics.hpp include paths
- Use full include path in water_isotopes header for robustness
Update all CMake references and include paths to use specialized_tracers
instead of aux_tracers for water tracer and isotope processes.
Intel oneAPI (ifx) fpp mishandles the COMMA macro-argument idiom used
throughout the MPAS framework (e.g. DMPAR_DEBUG_WRITE in mpas_dmpar.F),
miscounting a single macro argument as several and failing with
"number of arguments doesn't match". Preprocess .F files with GNU cpp
via GEN_F90=true instead, which handles the idiom correctly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Export per-PFT 2m reference temperature (t_ref2m) from ELM to IAC via
the coupler, and add the elm_ehc_deg_days namelist control:

ELM changes:
- lnd2iacMod.F90: Add t_ref2m(begg:endg,0:numpft) to lnd2iac_type;
  populate from veg_es%t_ref2m (instantaneous 2m temperature in K)
- elm_cpl_indices.F90: Add index_l2x_Sl_t_ref2m coupling index mapped
  to field name Sl_t_ref2m_topo##
- lnd_import_export.F90: Export lnd2iac_vars%t_ref2m into l2x vector

Coupler changes:
- seq_flds_mod.F90: Add Sl_t_ref2m_topo## to l2x_states (when
  add_iac_to_cplstate) and x2z_states with metadata (units: K)

GCAM namelist changes:
- namelist_definition_gcam.xml: Define elm_ehc_deg_days (logical,
  gcam_inparm group)
- namelist_defaults_gcam.xml: Default elm_ehc_deg_days to .true.
- build-namelist: Add elm_ehc_deg_days to default namelist generation

Update components/gcam/src submodule pointer.
Replace the approximate annual-mean t_ref2m-based heating/cooling degree day
computation with true sub-daily accumulation in ELM, wired through the full
ELM→coupler→IAC coupling chain.

Changes:
- lnd2iacMod.F90: Accumulate per-PFT HDD_accum and CDD_accum every ELM
  sub-daily timestep as max(t_base_K - t_ref2m, 0)*dt_days and
  max(t_ref2m - t_base_K, 0)*dt_days, where t_base_K = 291.15 K (18°C).
  Accumulators are monotonically increasing (never reset in ELM).
- elm_cpl_indices.F90: Add index_l2x_Sl_HDD_accum and index_l2x_Sl_CDD_accum
  per-PFT coupler index arrays; replace t_ref2m index.
- lnd_import_export.F90: Export HDD_accum/CDD_accum per PFT via the new
  coupler fields Sl_HDD_accum_pftNN / Sl_CDD_accum_pftNN.
- seq_flds_mod.F90: Register Sl_HDD_accum_pftNN and Sl_CDD_accum_pftNN in
  the l2x and x2z coupler state vectors; remove Sl_t_ref2m_topo fields.
Data flow schematic:

flowchart TD
    subgraph Sources["Data Sources"]
        FIRE["FireMod.F90\nforc_hdm\n(stream file / CPL_BYPASS)"]
        ATM["lnd2atm_vars\nt_ref2m_grc\n(2m reference temperature)"]
    end

    subgraph LND2IAC["lnd2iacMod.F90 - lnd2iac_type"]
        UPD["update_lnd2iac(bounds, lnd2atm_vars)\n* hr, npp, pftwgt  <- veg/col data\n* hdd = max(T_base - T_ref2m, 0)\n* cdd = max(T_ref2m - T_base, 0)\n  where T_base = 291.15 K (18 C)"]
        FIELDS["lnd2iac_vars fields:\n* hr(:,:)  * npp(:,:)  * pftwgt(:,:)\n* forc_hdm(:)  * hdd(:)  * cdd(:)"]
        UPD --> FIELDS
    end

    subgraph ELMDriver["elm_driver.F90"]
        DRIVER["lnd2iac_vars%update_lnd2iac\n(bounds_clump, lnd2atm_vars)"]
    end

    subgraph Export["lnd_import_export.F90 - lnd_export()"]
        COPY["Copy to coupler vector l2x:\n* forc_hdm <- FireMod::forc_hdm\n* hdd  <- lnd2iac_vars%hdd\n* cdd  <- lnd2iac_vars%cdd\n* hr, npp, pftwgt  (existing)"]
        IDX["elm_cpl_indices.F90\nindex_l2x_Sl_forc_hdm\nindex_l2x_Sl_hdd\nindex_l2x_Sl_cdd"]
        IDX --> COPY
    end

    subgraph CPL["driver-mct: seq_flds_mod.F90"]
        FLDS["Register coupler fields:\nSl_forc_hdm  (ind/km2)\nSl_hdd  (K-days)\nSl_cdd  (K-days)\nin l2x_states and x2z_states"]
    end

    subgraph GCAM["GCAM Component"]
        NML["Namelist flag:\nelm_ehc_hdd_cdd = .true.\n(namelist_defaults_gcam.xml)"]
        SRC["gcam/src submodule update\n(commit e034ca1)"]
        NML --> SRC
    end

    FIRE -->|forc_hdm public pointer| COPY
    ATM -->|t_ref2m_grc| UPD
    DRIVER -->|calls| UPD
    FIELDS -->|lnd2iac_vars%forc_hdm/hdd/cdd| COPY
    COPY -->|l2x vector| CPL
    FLDS -->|field metadata| CPL
    CPL -->|x2z coupler state| GCAM
HDD CDD transfer from ELM to GCAM
- Updated SMS test to SMS_Vmct.ne30pg2_f09_oEC60to30v3.SSP245_ZATM_BGC
- Updated ERS test to ERS_Vmct.ne30pg2_f09_oEC60to30v3.SSP245_ZATM_BGC
- These changes include new MOAB expectations needed for the two tests moving forward
Unit test for photo table.

Fixing yaml liking issue.

update values of etfphot_data.
This pull request makes a small change to how the ocean fraction variable is initialized in the `main()` function of 
`generate_domain_files_E3SM.py`. The code now initializes `ofrac` with ones instead of zeros, and uses the shape of 
`ds['omask']` rather than `ds['area_a']`.

* Changed initialization of `ofrac` to use ones with the shape of `ds['omask']` instead of zeros with the shape of
`ds['area_a']` in `generate_domain_files_E3SM.py`.
Only run unit test if nlev=72

Fixing warnings.

get yaml from server.

Update file name.
@xylar

xylar commented Aug 21, 2026

Copy link
Copy Markdown

Aurora Testing

(omega_dev) xylar@aurora-uan-0011:~/e3sm_work/omega/test_merge_e3sm_master/cime/scripts> ./create_test e3sm_omega_developer --wait
Using project from config_machines.xml: E3SM_Dec
create_test will do up to 4 tasks simultaneously
create_test will use up to 120 cores simultaneously
Creating test directory /lus/flare/projects/E3SM_Dec/xylar/scratch/ERS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu.20260821_125252_lvyjdb
Creating test directory /lus/flare/projects/E3SM_Dec/xylar/scratch/ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958.20260821_125252_lvyjdb
Creating test directory /lus/flare/projects/E3SM_Dec/xylar/scratch/PEM_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu.20260821_125252_lvyjdb
Creating test directory /lus/flare/projects/E3SM_Dec/xylar/scratch/SMS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu.20260821_125252_lvyjdb
Creating test directory /lus/flare/projects/E3SM_Dec/xylar/scratch/SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958.20260821_125252_lvyjdb
RUNNING TESTS:
  ERS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu
  ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958
  PEM_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu
  SMS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu
  SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958
Starting CREATE_NEWCASE for test ERS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu with 1 procs
Starting CREATE_NEWCASE for test ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958 with 1 procs
Starting CREATE_NEWCASE for test PEM_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu with 1 procs
Starting CREATE_NEWCASE for test SMS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu with 1 procs
Finished CREATE_NEWCASE for test ERS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu in 8.892456 seconds (PASS)
Finished CREATE_NEWCASE for test PEM_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu in 8.892162 seconds (PASS)
Starting XML for test ERS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu with 1 procs
Starting XML for test PEM_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu with 1 procs
Finished CREATE_NEWCASE for test SMS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu in 8.918909 seconds (PASS)
Starting XML for test SMS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu with 1 procs
Finished XML for test ERS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu in 0.407290 seconds (PASS)
Finished XML for test PEM_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu in 0.427293 seconds (PASS)
Starting SETUP for test ERS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu with 1 procs
Starting SETUP for test PEM_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu with 1 procs
Finished XML for test SMS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu in 0.424260 seconds (PASS)
Starting SETUP for test SMS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu with 1 procs
Finished CREATE_NEWCASE for test ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958 in 10.046614 seconds (PASS)
Starting XML for test ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958 with 1 procs
Finished XML for test ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958 in 0.338258 seconds (PASS)
Starting SETUP for test ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958 with 1 procs
Finished SETUP for test SMS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu in 5.536030 seconds (PASS)
Starting SHAREDLIB_BUILD for test SMS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu with 1 procs
Finished SETUP for test PEM_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu in 5.811716 seconds (PASS)
Finished SETUP for test ERS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu in 5.838565 seconds (PASS)
Starting SHAREDLIB_BUILD for test ERS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu with 1 procs
Starting SHAREDLIB_BUILD for test PEM_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu with 1 procs
Finished SETUP for test ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958 in 5.359813 seconds (PASS)
Starting SHAREDLIB_BUILD for test ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958 with 1 procs
Finished SHAREDLIB_BUILD for test ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958 in 166.970712 seconds (PASS)
Starting MODEL_BUILD for test ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958 with 11 procs
Finished SHAREDLIB_BUILD for test ERS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu in 167.510102 seconds (PASS)
Finished SHAREDLIB_BUILD for test SMS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu in 167.718950 seconds (PASS)
Starting MODEL_BUILD for test ERS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu with 11 procs
Starting MODEL_BUILD for test SMS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu with 11 procs
Finished SHAREDLIB_BUILD for test PEM_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu in 171.861235 seconds (PASS)
Starting MODEL_BUILD for test PEM_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu with 11 procs
Finished MODEL_BUILD for test ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958 in 184.482816 seconds (PASS)
Finished MODEL_BUILD for test ERS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu in 184.282187 seconds (PASS)
Finished MODEL_BUILD for test SMS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu in 184.285846 seconds (PASS)
Starting RUN for test ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958 with 1 proc on interactive node and 12 procs on compute nodes
Starting RUN for test ERS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu with 1 proc on interactive node and 12 procs on compute nodes
Starting RUN for test SMS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu with 1 proc on interactive node and 12 procs on compute nodes
Finished MODEL_BUILD for test PEM_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu in 180.094012 seconds (PASS)
Starting RUN for test PEM_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu with 1 proc on interactive node and 12 procs on compute nodes
Finished RUN for test ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958 in 10.804753 seconds (PEND). [COMPLETED 1 of 5]
Starting CREATE_NEWCASE for test SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958 with 1 procs
Finished CREATE_NEWCASE for test SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958 in 6.082475 seconds (PASS)
Starting XML for test SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958 with 1 procs
Finished XML for test SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958 in 0.339838 seconds (PASS)
Starting SETUP for test SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958 with 1 procs
Finished SETUP for test SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958 in 4.593915 seconds (PASS)
Starting SHAREDLIB_BUILD for test SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958 with 1 procs
Finished SHAREDLIB_BUILD for test SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958 in 164.336758 seconds (PASS)
Starting MODEL_BUILD for test SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958 with 11 procs
Finished RUN for test ERS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu in 254.910099 seconds (PEND). [COMPLETED 2 of 5]
Finished MODEL_BUILD for test SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958 in 162.815929 seconds (PASS)
Starting RUN for test SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958 with 1 proc on interactive node and 12 procs on compute nodes
Finished RUN for test SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958 in 1036.007581 seconds (PEND). [COMPLETED 3 of 5]
Finished RUN for test PEM_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu in 2178.867065 seconds (PEND). [COMPLETED 4 of 5]
Finished RUN for test SMS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu in 3090.147090 seconds (PEND). [COMPLETED 5 of 5]
Waiting for tests to finish
PASS ERS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu RUN
    Case dir: /lus/flare/projects/E3SM_Dec/xylar/scratch/ERS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu.20260821_125252_lvyjdb
PASS ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958 RUN
    Case dir: /lus/flare/projects/E3SM_Dec/xylar/scratch/ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958.20260821_125252_lvyjdb
PASS PEM_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu RUN
    Case dir: /lus/flare/projects/E3SM_Dec/xylar/scratch/PEM_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu.20260821_125252_lvyjdb
PASS SMS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu RUN
    Case dir: /lus/flare/projects/E3SM_Dec/xylar/scratch/SMS_Vmct.T62_oQU240.COMEGA-IAF.aurora_oneapi-ifxgpu.20260821_125252_lvyjdb
PASS SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958 RUN
    Case dir: /lus/flare/projects/E3SM_Dec/xylar/scratch/SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.aurora_oneapi-ifxgpu.omega-jra_1958.20260821_125252_lvyjdb
test-scheduler took 4264.647946834564 seconds

Polaris omega_pr suite

  • Baseline workdir: /lus/flare/projects/E3SM_Dec/xylar/polaris_0.10/aurora/test_20260821/omega-pr-cpu-develop
  • Baseline build: /lus/flare/projects/E3SM_Dec/xylar/polaris_0.10/aurora/test_20260821/omega-pr-cpu-develop/build
  • PR build: /lus/flare/projects/E3SM_Dec/xylar/polaris_0.10/aurora/test_20260821/omega-pr-cpu-test-merge-master/build
  • PR workdir: /lus/flare/projects/E3SM_Dec/xylar/polaris_0.10/aurora/test_20260821/omega-pr-cpu-test-merge-master
  • Machine: aurora
  • Compiler: oneapi-ifx
  • Build type: Release
  • Log: not found
  • Result:
    • Failures (1 of 22):
      • ocean/column/ekman

@sbrus89

sbrus89 commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator Author

@xylar, I've been using module load py-pyyaml/6.0.3 on Aurora to prevent the issue you saw. We may want to make an E3SM PR to load that automatically.

@xylar

xylar commented Aug 21, 2026

Copy link
Copy Markdown

Is that available on Chrysalis, too?

@sbrus89

sbrus89 commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator Author

@jonbob, since you were able to get the e3sm_omega_developer suite to pass in #511, do you have an idea of why @xylar is seeing fails?

@sbrus89

sbrus89 commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator Author

Is that available on Chrysalis, too?

Sorry, I'm not sure about Chrysalis.

@xylar

xylar commented Aug 21, 2026

Copy link
Copy Markdown

It doesn't look like py-pyyaml is available on Chrysalis so we'd need to install it or we'd need another fix.

@xylar

xylar commented Aug 21, 2026

Copy link
Copy Markdown

One option would be cime_env:

[xasaydavis@chrlogin1 ~]$ source /lcrc/soft/climate/e3sm-unified/load_latest_cime_env.sh 
loading compute pixi env...
   pixi env loaded.
(cime-env) [xasaydavis@chrlogin1 ~]$ python -c "import yaml"
(cime-env) [xasaydavis@chrlogin1 ~]$ 

This is a conda environment that is for CIME tests that require more than basic python (things like numpy). It clearly has pyyaml.

@jonbob

jonbob commented Aug 21, 2026

Copy link
Copy Markdown

I had added a command to load the python module on chrysalis that seemed to fix this -- on @andrewdnolan's testing PR

@xylar

xylar commented Aug 21, 2026

Copy link
Copy Markdown

@jonbob, are you referring to this line?

<command name="load">python/3.11.7-iexc6vq</command>

If so, that didn't help at least for me:

[xasaydavis@chrlogin1 ~]$ module load python/3.11.7-iexc6vq
[xasaydavis@chrlogin1 ~]$ python -c "import yaml"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'yaml'

@jonbob

jonbob commented Aug 21, 2026

Copy link
Copy Markdown

@xylar -- that's really weird. It works if I do the same thing:

[ac.jwolfe@chrlogin1 ~]$ module load python/3.11.7-iexc6vq
[ac.jwolfe@chrlogin1 ~]$ python -c "import yaml"
[ac.jwolfe@chrlogin1 ~]$ 

@xylar

xylar commented Aug 21, 2026

Copy link
Copy Markdown

@jonbob, do you have another python already loaded before that (e.g. in your .bashrc)? I have:

module load git
module load python

but the latter just loads the same python above:

$ which python
/gpfs/fs1/soft/chrysalis/spack-latest/opt/spack/linux-rhel8-x86_64/gcc-11.3.0/python-3.11.7-iexc6vq/bin/python
ls /gpfs/fs1/soft/chrysalis/spack-latest/opt/spack/linux-rhel8-x86_64/gcc-11.3.0/python-3.11.7-iexc6vq/lib/python3.11/

shows no yaml that I can see.

@xylar

xylar commented Aug 21, 2026

Copy link
Copy Markdown

By the way, my test failures above seem to be because of the same:

$ vim /lcrc/group/e3sm/xasaydavis/scratch/chrys/ERS_Vmct.T62_oQU240.COMEGA-IAF.chrysalis_intel.20260821_074556_wci8md/TestStatus.log

2026-08-21 07:53:00: Exception during run:
No module named 'yaml'
Traceback (most recent call last):

@jonbob

jonbob commented Aug 21, 2026

Copy link
Copy Markdown

@xylar -- I just have

module load python

in my .bashrc as well

…nd (PR #8651)

For pm-gpu, reapply a LD_LIBRARY_PATH work-around to avoid cuda runtime error
@jonbob

jonbob commented Aug 21, 2026

Copy link
Copy Markdown

But I ran a test with

print(yaml.__file__)

and it does point to a local python:

/home/ac.jwolfe/.local/lib/python3.11/site-packages/yaml/__init__.py

So apparently loading the module is not a fix

@xylar

xylar commented Aug 21, 2026

Copy link
Copy Markdown

@jonbob, yes, I think you did something in your

/home/ac.jwolfe/.local/bin/
/home/ac.jwolfe/.local/lib/

and that's why it works for you but that won't help the rest of us, I'm afraid.

@sbrus89

sbrus89 commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator Author

I see similar behavior to @jonbob:

sbrus@chrlogin1:~$ python -c "import yaml; print(yaml.__file__)"
/home/sbrus/.local/lib/python3.11/site-packages/yaml/__init__.py

So I must have installed yaml in my home directory as well somehow.

@xylar

xylar commented Aug 21, 2026

Copy link
Copy Markdown

Having each developer install pyyaml locally really isn't a good solution in my opinion. It's very messy. We need a clean alternative.

@xylar

xylar commented Aug 21, 2026

Copy link
Copy Markdown

@rljacob, do you have thoughts?

@sbrus89

sbrus89 commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator Author

Passes e3sm_omega_developer on pm-cpu and pm-gpu:

20260821_114539_3twcb7: 5 tests
  ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-gpu_gnugpu.omega-jra_1958 (Overall: FAIL) details:
    PASS ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-gpu_gnugpu.omega-jra_1958 CREATE_NEWCASE
    PASS ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-gpu_gnugpu.omega-jra_1958 XML
    PASS ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-gpu_gnugpu.omega-jra_1958 SETUP
    PASS ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-gpu_gnugpu.omega-jra_1958 SHAREDLIB_BUILD time=92
    PASS ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-gpu_gnugpu.omega-jra_1958 MODEL_BUILD time=177
    PASS ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-gpu_gnugpu.omega-jra_1958 SUBMIT
    FAIL ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-gpu_gnugpu.omega-jra_1958 RUN time=1813
  ERS_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu (Overall: PASS) details:
    PASS ERS_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu CREATE_NEWCASE
    PASS ERS_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu XML
    PASS ERS_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu SETUP
    PASS ERS_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu SHAREDLIB_BUILD time=93
    PASS ERS_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu MODEL_BUILD time=180
    PASS ERS_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu SUBMIT
    PASS ERS_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu RUN time=1349
    PASS ERS_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu COMPARE_base_rest
    PASS ERS_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu MEMLEAK
    PASS ERS_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu SHORT_TERM_ARCHIVER
  PEM_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu (Overall: PASS) details:
    PASS PEM_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu CREATE_NEWCASE
    PASS PEM_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu XML
    PASS PEM_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu SETUP
    PASS PEM_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu SHAREDLIB_BUILD time=106
    PASS PEM_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu MODEL_BUILD time=173
    PASS PEM_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu SUBMIT
    PASS PEM_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu RUN time=1223
    PASS PEM_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu COMPARE_base_modpes
    PASS PEM_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu MEMLEAK
    PASS PEM_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu SHORT_TERM_ARCHIVER
  SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-gpu_gnugpu.omega-jra_1958 (Overall: PASS) details:
    PASS SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-gpu_gnugpu.omega-jra_1958 CREATE_NEWCASE
    PASS SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-gpu_gnugpu.omega-jra_1958 XML
    PASS SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-gpu_gnugpu.omega-jra_1958 SETUP
    PASS SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-gpu_gnugpu.omega-jra_1958 SHAREDLIB_BUILD time=92
    PASS SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-gpu_gnugpu.omega-jra_1958 MODEL_BUILD time=181
    PASS SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-gpu_gnugpu.omega-jra_1958 SUBMIT
    PASS SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-gpu_gnugpu.omega-jra_1958 RUN time=1459
    PASS SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-gpu_gnugpu.omega-jra_1958 MEMLEAK insufficient data for memleak test
    PASS SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-gpu_gnugpu.omega-jra_1958 SHORT_TERM_ARCHIVER
  SMS_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu (Overall: PASS) details:
    PASS SMS_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu CREATE_NEWCASE
    PASS SMS_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu XML
    PASS SMS_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu SETUP
    PASS SMS_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu SHAREDLIB_BUILD time=90
    PASS SMS_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu MODEL_BUILD time=146
    PASS SMS_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu SUBMIT
    PASS SMS_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu RUN time=442
    PASS SMS_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu MEMLEAK
    PASS SMS_Vmct.T62_oQU240.COMEGA-IAF.pm-gpu_gnugpu SHORT_TERM_ARCHIVER
20260821_115941_6546bk: 5 tests
  ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-cpu_gnu.omega-jra_1958 (Overall: PASS) details:
    PASS ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-cpu_gnu.omega-jra_1958 CREATE_NEWCASE
    PASS ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-cpu_gnu.omega-jra_1958 XML
    PASS ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-cpu_gnu.omega-jra_1958 SETUP
    PASS ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-cpu_gnu.omega-jra_1958 SHAREDLIB_BUILD time=90
    PASS ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-cpu_gnu.omega-jra_1958 MODEL_BUILD time=158
    PASS ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-cpu_gnu.omega-jra_1958 SUBMIT
    PASS ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-cpu_gnu.omega-jra_1958 RUN time=433
    PASS ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-cpu_gnu.omega-jra_1958 COMPARE_base_rest
    PASS ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-cpu_gnu.omega-jra_1958 MEMLEAK insufficient data for memleak test
    PASS ERS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-cpu_gnu.omega-jra_1958 SHORT_TERM_ARCHIVER
  ERS_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu (Overall: PASS) details:
    PASS ERS_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu CREATE_NEWCASE
    PASS ERS_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu XML
    PASS ERS_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu SETUP
    PASS ERS_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu SHAREDLIB_BUILD time=91
    PASS ERS_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu MODEL_BUILD time=158
    PASS ERS_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu SUBMIT
    PASS ERS_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu RUN time=281
    PASS ERS_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu COMPARE_base_rest
    PASS ERS_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu MEMLEAK
    PASS ERS_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu SHORT_TERM_ARCHIVER
  PEM_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu (Overall: PASS) details:
    PASS PEM_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu CREATE_NEWCASE
    PASS PEM_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu XML
    PASS PEM_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu SETUP
    PASS PEM_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu SHAREDLIB_BUILD time=101
    PASS PEM_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu MODEL_BUILD time=148
    PASS PEM_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu SUBMIT
    PASS PEM_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu RUN time=231
    PASS PEM_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu COMPARE_base_modpes
    PASS PEM_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu MEMLEAK
    PASS PEM_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu SHORT_TERM_ARCHIVER
  SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-cpu_gnu.omega-jra_1958 (Overall: PASS) details:
    PASS SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-cpu_gnu.omega-jra_1958 CREATE_NEWCASE
    PASS SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-cpu_gnu.omega-jra_1958 XML
    PASS SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-cpu_gnu.omega-jra_1958 SETUP
    PASS SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-cpu_gnu.omega-jra_1958 SHAREDLIB_BUILD time=90
    PASS SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-cpu_gnu.omega-jra_1958 MODEL_BUILD time=158
    PASS SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-cpu_gnu.omega-jra_1958 SUBMIT
    PASS SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-cpu_gnu.omega-jra_1958 RUN time=290
    PASS SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-cpu_gnu.omega-jra_1958 MEMLEAK insufficient data for memleak test
    PASS SMS_Vmct_Ln5.TL319_EC30to60E2r2.COMEGA-JRA1p5.pm-cpu_gnu.omega-jra_1958 SHORT_TERM_ARCHIVER
  SMS_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu (Overall: PASS) details:
    PASS SMS_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu CREATE_NEWCASE
    PASS SMS_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu XML
    PASS SMS_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu SETUP
    PASS SMS_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu SHAREDLIB_BUILD time=89
    PASS SMS_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu MODEL_BUILD time=142
    PASS SMS_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu SUBMIT
    PASS SMS_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu RUN time=117
    PASS SMS_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu MEMLEAK
    PASS SMS_Vmct.T62_oQU240.COMEGA-IAF.pm-cpu_gnu SHORT_TERM_ARCHIVER

@sbrus89

sbrus89 commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator Author

Passes CTests and omega_pr with gnugpu on pm-gpu.

Polaris omega_pr suite

  • Baseline workdir: /global/homes/s/sbrus/scratch/polaris_baseline_omega_pr_omega_gpu/
  • Baseline build: /global/homes/s/sbrus/scratch/polaris_baseline_omega_pr_omega_gpu/build
  • PR build: /global/homes/s/sbrus/scratch/polaris_e3sm_merge_omega_pr_omega_gpu/build
  • PR workdir: /global/homes/s/sbrus/scratch/polaris_e3sm_merge_omega_pr_omega_gpu
  • Machine: pm-gpu
  • Compiler: gnugpu
  • Build type: Release
  • Log: /global/homes/s/sbrus/scratch/polaris_e3sm_merge_omega_pr_omega_gpu/polaris_omega_pr.o57380767
  • Result:
    • Failures (1 of 22):
      • ocean/column/ekman

Note that the path workaround: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CRAY_LD_LIBRARY_PATH from E3SM-Project#8651 is needed prior to running CTests or omega_pr.

jgfouca and others added 3 commits August 21, 2026 15:36
Just for testing auto blesser

For testing purposes only.

[BFB]
We are using the DataInterpolation class for surface emissions. When this code
was originally written, we used the former SAP reader as a guide to implement
the surface-emission reader. As a result, there is a discrepancy in how time
interpolation is performed. In particular, the former SAP reader uses the
variable date instead of the time variable, and the day in the date is not
taken into account when selecting the time interval. These differences produce
discrepancies in the mam4xx tests.

[non-BFB]
@sbrus89

sbrus89 commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator Author

@xylar, E3SM-Project#8632 is still undergoing testing so I'm going to merge this since we've tested it already. I think we need to bring in the pm-gpu workaround now so we can test there. We can do another master merge once Kokkos is updated in E3SM. That one may not be BFB anyway, so it will be good to isolate that change and do a polaris submodule update.

@sbrus89
sbrus89 merged commit 26ee34e into E3SM-Project:develop Aug 25, 2026
sbrus89 added a commit that referenced this pull request Aug 27, 2026
There seems to have been a minor mistake in resolving the diffs from #521,
which resulted in the e3sm_omega_developer test suite definition missing it's
closing bracket.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.