Changes to CPLHIST mode for robustness and getting the right default settings for CLM and an updated 1850 ndep file by ekluzek · Pull Request #403 · ESCOMP/CDEPS

ekluzek · 2026-04-15T20:51:11Z

Description of changes

Work with 1850_clim settings for ndep. As well as some work with CPLHIST options.

Specific notes

Contributors other than yourself, if any: @billsacks

CDEPS Issues Fixed (include github issue #):

Fixes Add error checking and set defaults for DATM_CPLHIST_DIR and DATM_CPLHIST_CASE #399
Fixes Bring in option for the new preliminary CMIP7 1850 ndep dataset #388
Fixes Add expect statements to the DATM buildnml for the NEON/PLUMBER2/generic tower settings #432
Fixes Update CPLHIST data for DATM for cesm3 spinup with preliminary data #397
Fixes New DATM CMIP7 CO2 file replacing earlier one that would fail after 6 month simulation #410

Are there dependencies on other component PRs (if so list):

Are changes expected to change answers (bfb, different to roundoff, more substantial): No
This will add some new options, but won't change defaults (which primarily need to be changed in compsets anyway)

Any User Interface Changes (namelist or namelist defaults changes): Yes

Testing performed (e.g. aux_cdeps, CESM prealpha, etc): Have tested a few cases will test aux_cdeps against cesm3_0_beta08 tests

Hashes used for testing:

Definition of done:

Finish work on all of the modes
Add needed aux_cdeps tests
Test against the aux_cdeps cesm3_0_beta09 baseline and make sure identical
Test in the context of CTSM and make sure it does everything needed

…o CLM

… directory location for the DATM_CPLHIST_CASE under cpl/hist subdirectory

cdeps1.0.94

wwieder · 2026-05-20T16:02:16Z

It's not clear to me if the intent of this PR + #405 are needed for:

CLM to be able to use CPL_HIST output (and therefore lower priority for an alpha tag); or
CESM to write out CPL_HIST files that we'll use for spinup (and therefore a higher priority)?

billsacks · 2026-05-20T16:34:39Z

It's not clear to me if the intent of this PR + #405 are needed for:

CLM to be able to use CPL_HIST output (and therefore lower priority for an alpha tag); or

CESM to write out CPL_HIST files that we'll use for spinup (and therefore a higher priority)?

It's (1) - to be able to use CPLHIST output.

wwieder · 2026-05-21T15:08:44Z

OK, if this is only needed for CTSM to read CPL_HIST data my read is that this PR isn't critical for the next alpha09 tag. What do others think?

cdeps1.0.105 Conflicts: datm/cime_config/config_component.xml

…g more

…w including important points about clm_usr_dat_name and available_neon_data

…and use it to define it, and also check that if one is set the other is as well, and then ensure single_column is set at the end of the subroutine

…ear 1 for CPLHIST spinup cases

ekluzek · 2026-08-25T17:38:09Z

OK, I did the things from the code review from @slevis-lmwg. I'm going to redo testing, and verifty things are expected, and then I'd like to merge it, so I can have a tag for CTSM.

ekluzek · 2026-08-25T17:55:47Z

OK, I am sending testing for aux_clm and aux_cdeps and will want to merge once that testing is complete.

billsacks

I have a lot of specific comments here, but by and large I really appreciate all of the work you've done to both get cplhist mode working and also to do some cleanup and add some error checking to the datm buildnml! Thank you!!

billsacks · 2026-08-25T16:10:23Z

+      <var>atmImp_Faxa_ndep1  Faxa_ndep_nhx</var>
+      <var>atmImp_Faxa_ndep2  Faxa_ndep_noy</var>


I want to confirm this: It looks like the previous version of this was set up for cpl7-based cplhist, so I see the need for this change. It looks like this triggers the cmip6 (as opposed to use_cmip7_ndep) block in the Fortran. It looks like this leads to the following block in datm_pres_ndep_advance:

! convert ndep flux to units of kgN/m2/s (input is in gN/m2/s) Faxa_ndep(1,:) = strm_Faxa_ndep_nhx(:) / 1000._r8 Faxa_ndep(2,:) = strm_Faxa_ndep_noy(:) / 1000._r8

Does that look correct - in particular this unit conversion? If not, we'll have to disambiguate these cases.

Oh, this is an important catch! Thanks for noticing that!

The check should really be on the units in the files, rather than the number of fields to indicate if it's cmip7 or cmip6.

I made this into an issue #436

I have several questions:

when did the filenames_advance get added - and what does it do?

what is the solution to Read in the units for variables on the stream files #436? Is there new code to be added to check the units? Will that be part of this PR?

@billsacks nice catch! Thanks for your careful review.
@ekluzek is planning on bring in the units check for NDEP in a different PR

I have several questions:

when did the filenames_advance get added - and what does it do?

It looks like it was added 3 years ago by @billsacks and @jedwards4b.

It indicates that the data starts with day 2, rather than 1.

Here's the documentation from the code:

filename_advance_days is an integer specifying the number of days to add to the date portion of the file name when using %ymd. Currently only values of 0 or 1 are supported. This is typically 0 but can be 1 to indicate that the dates have a one-day offset, starting with day 2 in the first year and ending with day 1 in the year following the last year. This can be the case, for example, for daily coupler history files.

It along with %ymd allows you to use the raw CPLHIST files -- without having to do post-processing to put them into monthly or yearly files. So I think this is really helpful.

what is the solution to Read in the units for variables on the stream files #436? Is there new code to be added to check the units? Will that be part of this PR?

As @wwieder said I'd like to do that in a seperate follow on PR. That does mean that NDEP for CLM for CPLHIST is off by 1000X, but I'd like to get this part in first before working on that issue. I'm not quite sure what fixing that will take, so figure it should be seperate.

And while you are asking. Currently this PR contains changes that are both answer changing and ones that aren't. There are somewhat tied together, but I could maybe look into seperating out the b4b parts and make that the first PR to bring in. @billsacks and @mvertens do you have a preference on that? And is doing a following on PR OK here?

billsacks · 2026-08-25T16:25:28Z

+      <!-- Alturnative form to use:
+      the following stream fields are in units of kg/m2/sec - so need no unit conversion
+      <var>dry_deposition_NHx_as_N Faxa_ndep_nhx_dry</var>
+      <var>wet_deposition_NHx_as_N Faxa_ndep_nhx_wet</var>
+      <var>dry_deposition_NOy_as_N Faxa_ndep_noy_dry</var>
+      <var>wet_deposition_NOy_as_N Faxa_ndep_noy_wet</var>
+      -->


It's not clear to me why you have this commented-out alternative block. Is this needed for some files? Can you either clarify in the comment why you might want to switch to this or else just remove it?

Also, spelling error: Alturnative -> Alternative.

OK, I suppose there isn't a reason to have these here. You can either give the 4 fields or the 2 and either will work. And they should be identical if you ignore file precision. If the data is single precision it's probably more accurate to use just the two fields because the sum would've been done in double and converted to single. So I'll just delete the comment.

This is actually similar to other things in the code where there are multiple ways of entering the same bits of data. I'm just kind of highlighting that possibility here. But, this should be pointed out elsewhere.

So I'll remove the comment.

Actually based on #436 the comment is the right way to do it, and the code I have in place is wrong.

I did remove the commented out code. But, this means the answers are wrong, because it thinks it's CMIP6 units.

billsacks · 2026-08-25T16:53:08Z

    <file>env_run.xml</file>
-    <desc>DATM CO2 time series</desc>
+    <desc>DATM CO2 time series
+          NOTE: This needs to be coordinated with CCSM_BGC in CMEPS, and wtih either the ocean or land model


Typo: wtih -> with

Resolved in 7d61189

billsacks · 2026-08-25T16:53:50Z

      <value compset="^OMIP_DATM%IAF.*_POP2%[^_]*ECO">omip.iaf</value>
      <value compset="^OMIP_DATM%JRA.*_POP2%[^_]*ECO">omip.jra</value>
-      <value compset="_DATM%CPLHIST">cplhist</value>
+      <value compset="_DATM%CPLHIST_CLM">cplhist</value>


What is the rationale for this change - i.e., only setting cplhist mode for compsets with CLM?

If this is kept, the regex should be changed to "_DATM%CPLHIST.*_CLM"

I actually want to have it set all the time, as we want to be for both MOM and CLM. But, I realized that I should be more restrictive now until something more general can happen (like ESCOMP/CMEPS#698). MOM will need to coordinate with this, so we should change for it, only when they are ready to go.

I'll be talking with @mnlevy1981 and @klindsay28 about how best to coordinate this though. So I expect at least a change here saying what to do for MOM, and maybe later a more general thing going in.

Just clarifying that the CPLHIST_CLM compsets are only for this intermediate PR, until we get CPLHIST for MOM set so both land and ocean can use the same CPLHIST datasets?

@wwieder yes that is correct. This sets it up for CLM. Then MOM will need to make a PR to CDEPS that they include in a MOM tag. At that point we can both use CPLHIST mode.

I think a future unification step should be done later, so that the same line works for both CLM and MOM. But, that isn't immediately necessary.

I'll also be talking to Ocean folks about this as well, to make sure it works for everyone.

billsacks · 2026-08-25T22:54:27Z

+        expect(False, "If either PTS_LON or PTS_LAT is set, the other must be set as well")
+
+    # Verify that the expected output setting was actually done
+    expect(config['single_column'] is not None, "single_column should have been set in this subroutine")


It might feel kind of arbitrary, but I also feel like this expect is helpful. It's hard for me to explain my thinking on this one vs. the other ones in _handle_model_grid. Maybe the difference is that this is a single, simple expect that feels like it's doing a bigger job, and also less likely to need changes if you change the production code, partly because the expect is unconditional, so you don't need to replicate logic from earlier in the routine in order to do this expect? But I do acknowledge that this all feels subjective, so I'm open to disagreements with my feelings on where there should / shouldn't be expects.

However: looking at the above logic, it looks like config['single_column'] isn't set if case.get_value('PTS_DOMAINFILE') gives None or an empty string. So maybe you need to add to the logic to make this expect always pass?

Yeah, I think you are right. The strange thing is that this should have been caught in the NEON/PLUMBER2 tests in aux_clm. So I'll look at why those are working. Or perhaps I didn't retest after adding the expect?

Ahhh, yes this only fails if PTS_DOMAINFILE doesn't exist or is empty. So tests pass because this is defined in CMEPS so all configurations have it available and by default set to UNSET.

But, for future proofing I think it should abort if PTS_DOMAINFILE isn't defined and just say that it isn't. Now, single single_column is a special case (and not the typical one for CESM), that should probably be allowed unless PTS_LON/PTS_LAT are set. So I'll do something along the lines of that second option...

billsacks · 2026-08-25T23:06:16Z

+        expect(cplhist_case is not None, "DATM_CPLHIST_CASE must be set when using cplhist mode")
+        expect(cplhist_case != "UNSET", "DATM_CPLHIST_CASE must be set when using cplhist mode")


Minor nitpick: I would rewrite these two separate expect statements into a single expect statement based on cplhist_case is not None and cplhist_case != "UNSET": I find it clearer when reading the code to have a single expect on a variable when possible. (I read the first expect and was initially confused about why you were checking against None rather than UNSET... then realized that you had two expects for this variable.)

I do appreciate the expects ensuring that these various DATM_CPLHIST variables are set when they should be!!

billsacks · 2026-08-25T23:08:56Z

+        expect(cplhist_case is not None, "DATM_CPLHIST_CASE must be set when using cplhist mode")
+        expect(cplhist_case != "UNSET", "DATM_CPLHIST_CASE must be set when using cplhist mode")
+        expect(cplhist_dir is not None, "DATM_CPLHIST_DIR must be set when using cplhist mode")
+        expect(os.path.isdir(cplhist_dir), "DATM_CPLHIST_DIR {} does not exist".format(cplhist_dir))


Tied in with my comment elsewhere about preferring a default value of UNSET for DATM_CPLHIST_DIR: that would make this error check more straightforward (you could just check to make sure that it isn't still UNSET).

I don't love the check of os.path.isdir here: I appreciate the intent, but it feels too early in the process to check this: I could imagine someone setting up a case and running preview_namelists before setting up the directory structure that actually contains the cplhist forcings. It may be unlikely, but it seems like that should be an acceptable order and we shouldn't force the directory to already exist at preview_namelists time. But I could be convinced otherwise, particularly if we already enforce directory existence at preview_namelists time in other places.

The problem here is that this is to avoid having check_input_data return a list of missing files that aren't easy for the user to decipher. It'll look like this and go on forever...

/glade/campaign/cesm/cesmdata/inputdata/cplhist/cesm3_0/$DATM_CPLHIST_CASE/atm/$DATM_CPLHIST_CASE.cpl.hx.atm.3hprec.avrg.0101-01-01-10800.nc

when this was because $DATM_CPLHIST_CASE was set to UNSET it was hard to decipher. But, now that you HAVE to set the casename the above might be acceptable.

So you mgiht be right that this should/could be removed.

The main thing is that I'd rather be told that the directory doesn't exist -- then see the long list of files that don't exist. I think I prefer to be told that early rather than thinking I have things setup when I really don't because the directory doesn't even exist.

I added some comments around this. And also added writing it out unresolved so you know what variables might need to change.

billsacks · 2026-08-25T23:10:05Z

+        expect(cplhist_dir is not None, "DATM_CPLHIST_DIR must be set when using cplhist mode")
+        expect(os.path.isdir(cplhist_dir), "DATM_CPLHIST_DIR {} does not exist".format(cplhist_dir))
+        if cplhist_domain != "null":
+           expect(os.path.isfile(cplhist_domain), "DATM_CPLHIST_DOMAIN_FILE {} does not exist".format(cplhist_domain))


As with my comment for DATM_CPLHIST_DIR, I feel like preview_namelists might be too early to do this file existence check. Do we do file existence checks elsewhere at preview_namelists time? (If so, then I stand corrected, so ignore this comment.)

For this one you are right, I should let check_input_data check for existence of this file.

I did check and we do check for file and directory existence, as seen below. But, it's mostly for things that would always be there (the RUNDIR, CONFDIR, namelist xml files etc.). But, there is checking for things like the user_mod directory files and rpointer files as well.

erik/cesm_alpha2> find . -name buildnml -print -exec grep isdir {} \; ./components/mizuRoute/cime_config/buildnml if not os.path.isdir(confdir): if os.path.isdir(rundir): ./components/mom/cime_config/buildnml ./components/rtm/cime_config/buildnml if not os.path.isdir(confdir): expect(os.path.isdir(user_xml_dir), if os.path.isdir(rundir): ./components/ww3/cime_config/buildnml if not os.path.isdir(confdir): expect(os.path.isdir(user_xml_dir), if os.path.isdir(rundir): if os.path.isdir(input_dir): ./components/clm/cime_config/buildnml if not os.path.isdir(ctsmconf): if os.path.isdir(rundir): ./components/mosart/cime_config/buildnml if not os.path.isdir(confdir): expect(os.path.isdir(user_xml_dir), if os.path.isdir(rundir): ./components/cdeps/dwav/cime_config/buildnml if not os.path.isdir(confdir): expect (os.path.isdir(user_xml_dir), ./components/cdeps/dglc/cime_config/buildnml if not os.path.isdir(confdir): expect (os.path.isdir(user_xml_dir), ./components/cdeps/dlnd/cime_config/buildnml if not os.path.isdir(confdir): expect (os.path.isdir(user_xml_dir), ./components/cdeps/docn/cime_config/buildnml if not os.path.isdir(confdir): expect (os.path.isdir(user_xml_dir), ./components/cdeps/drof/cime_config/buildnml if not os.path.isdir(confdir): expect (os.path.isdir(user_xml_dir), ./components/cdeps/dice/cime_config/buildnml if not os.path.isdir(confdir): expect (os.path.isdir(user_xml_dir), ./components/cdeps/datm/cime_config/buildnml expect(os.path.isdir(cplhist_dir), "DATM_CPLHIST_DIR {} does not exist".format(cplhist_dir)) if not os.path.isdir(confdir): expect (os.path.isdir(user_xml_dir), ./components/cam/cime_config/buildnml if os.path.isdir(rundir): if os.path.isdir(rundir) \ ./components/cmeps/cime_config/buildnml if not os.path.isdir(confdir): os.path.isdir(user_xml_dir), "user_xml_dir %s does not exist " % user_xml_dir ./components/cmeps/med_test_comps/xice/cime_config/buildnml ./components/cmeps/med_test_comps/xlnd/cime_config/buildnml ./components/cmeps/med_test_comps/xrof/cime_config/buildnml ./components/cmeps/med_test_comps/xwav/cime_config/buildnml ./components/cmeps/med_test_comps/xglc/cime_config/buildnml ./components/cmeps/med_test_comps/xatm/cime_config/buildnml ./components/cmeps/med_test_comps/xocn/cime_config/buildnml ./components/cice/cime_config/buildnml if not os.path.isdir(confdir): expect (os.path.isdir(user_xml_dir), if os.path.isdir(rundir): ./components/cism/cime_config/buildnml if not os.path.isdir(confdir): if os.path.isdir(rundir): if os.path.isdir(rundir): expect (os.path.isdir(user_xml_dir), ./cime/CIME/non_py/src/components/stub_comps_nuopc/sglc/cime_config/buildnml ./cime/CIME/non_py/src/components/stub_comps_nuopc/srof/cime_config/buildnml ./cime/CIME/non_py/src/components/stub_comps_nuopc/siac/cime_config/buildnml ./cime/CIME/non_py/src/components/stub_comps_nuopc/sesp/cime_config/buildnml ./cime/CIME/non_py/src/components/stub_comps_nuopc/swav/cime_config/buildnml ./cime/CIME/non_py/src/components/stub_comps_nuopc/sice/cime_config/buildnml ./cime/CIME/non_py/src/components/stub_comps_nuopc/slnd/cime_config/buildnml ./cime/CIME/non_py/src/components/stub_comps_nuopc/socn/cime_config/buildnml ./cime/CIME/non_py/src/components/stub_comps_nuopc/satm/cime_config/buildnml erik/cesm_alpha2> find . -name buildnml -print -exec grep isfile {} \; ./components/mizuRoute/cime_config/buildnml expect(os.path.isfile(user_nl_file), ./components/mom/cime_config/buildnml ./components/rtm/cime_config/buildnml if os.path.isfile(user_definition): expect(os.path.isfile(file_), "Namelist XML file %s not found!" % file_) if (os.path.isfile(os.path.join(rundir, rpointer)) and (not os.path.isfile(os.path.join(rundir, rpointer + inst_string)))): expect(os.path.isfile(user_nl_file), ./components/ww3/cime_config/buildnml if (os.path.isfile(os.path.join(rundir, base)) and not os.path.isfile(os.path.join(rundir, base + ".nc"))): if os.path.isfile(user_definition): expect(os.path.isfile(file_), "Namelist XML file %s not found!" % file_) if (os.path.isfile(os.path.join(rundir, rpointer)) and (not os.path.isfile(os.path.join(rundir, rpointer + inst_string)))): expect(os.path.isfile(user_nl_file), if os.path.isfile(bld_ww3_grid): if not os.path.isfile(os.path.join(output_dir, filename)): if not os.path.isfile(os.path.join(output_dir,"ww3_grid")): elif not os.path.isfile(os.path.join(output_dir,"ww3_grid.inp")): if not os.path.isfile(os.path.join(output_dir,"mod_def.ww3")): if os.path.isfile(mod_def_in): if not os.path.isfile(os.path.join(rundir, "mod_def.ww3")): ./components/clm/cime_config/buildnml if os.path.isfile(os.path.join(rundir, rpointer)) and ( not os.path.isfile(os.path.join(rundir, rpointer + inst_string)) ./components/mosart/cime_config/buildnml if os.path.isfile(user_definition): expect(os.path.isfile(file_), "Namelist XML file %s not found!" % file_) if (os.path.isfile(os.path.join(rundir, rpointer)) and (not os.path.isfile(os.path.join(rundir, rpointer + inst_string)))): expect(os.path.isfile(user_nl_file), ./components/cdeps/dwav/cime_config/buildnml if os.path.isfile(user_definition): expect(os.path.isfile(file_), "Namelist XML file {} not found!".format(file_)) if (os.path.isfile(os.path.join(rundir,rpointer)) and (not os.path.isfile(os.path.join(rundir,rpointer + inst_string)))): expect(os.path.isfile(user_nl_file), ./components/cdeps/dglc/cime_config/buildnml if os.path.isfile(user_definition): expect(os.path.isfile(file_), "Namelist XML file {} not found!".format(file_)) if (os.path.isfile(os.path.join(rundir,rpointer)) and (not os.path.isfile(os.path.join(rundir,rpointer + inst_string)))): expect(os.path.isfile(user_nl_file), ./components/cdeps/dlnd/cime_config/buildnml if os.path.isfile(user_definition): expect(os.path.isfile(file_), "Namelist XML file {} not found!".format(file_)) if (os.path.isfile(os.path.join(rundir,rpointer)) and (not os.path.isfile(os.path.join(rundir,rpointer + inst_string)))): expect(os.path.isfile(user_nl_file), ./components/cdeps/docn/cime_config/buildnml if os.path.isfile(user_definition): expect(os.path.isfile(file_), "Namelist XML file {} not found!".format(file_)) if (os.path.isfile(os.path.join(rundir,rpointer)) and (not os.path.isfile(os.path.join(rundir,rpointer + inst_string)))): expect(os.path.isfile(user_nl_file), ./components/cdeps/drof/cime_config/buildnml if os.path.isfile(user_definition): expect(os.path.isfile(file_), "Namelist XML file {} not found!".format(file_)) if (os.path.isfile(os.path.join(rundir,rpointer)) and (not os.path.isfile(os.path.join(rundir,rpointer + inst_string)))): expect(os.path.isfile(user_nl_file), ./components/cdeps/dice/cime_config/buildnml if os.path.isfile(user_definition): expect(os.path.isfile(file_), "Namelist XML file {} not found!".format(file_)) if (os.path.isfile(os.path.join(rundir,rpointer)) and (not os.path.isfile(os.path.join(rundir,rpointer + inst_string)))): expect(os.path.isfile(user_nl_file), ./components/cdeps/datm/cime_config/buildnml expect(os.path.isfile(cplhist_domain), "DATM_CPLHIST_DOMAIN_FILE {} does not exist".format(cplhist_domain)) if os.path.isfile(user_definition): expect(os.path.isfile(file_), "Namelist XML file {} not found!".format(file_)) if (os.path.isfile(os.path.join(rundir,rpointer)) and (not os.path.isfile(os.path.join(rundir,rpointer + inst_string)))): expect(os.path.isfile(user_nl_file), ./components/cam/cime_config/buildnml if not os.path.isfile(filename): if os.path.isfile(file2) and not filecmp.cmp(file1, file2): expect(os.path.isfile(filename), if os.path.isfile(input_data_list): if os.path.isfile(single_case_rpointer) and \ not os.path.isfile(instance_rpointer): if not os.path.isfile(os.path.join(rundir, ncdata)): if not os.path.isfile(os.path.join(rundir, cam_branch_file)): if (os.path.isfile(file1)) and (not os.path.isfile(file2)): ./components/cmeps/cime_config/buildnml if os.path.isfile(comp_config_file): if os.path.isfile(infile): esmfmkfile and os.path.isfile(esmfmkfile), if os.path.isfile(user_drv_definition): if os.path.isfile(drv_flds_in): if os.path.isfile(user_yaml_file): ./components/cmeps/med_test_comps/xice/cime_config/buildnml ./components/cmeps/med_test_comps/xlnd/cime_config/buildnml ./components/cmeps/med_test_comps/xrof/cime_config/buildnml ./components/cmeps/med_test_comps/xwav/cime_config/buildnml ./components/cmeps/med_test_comps/xglc/cime_config/buildnml ./components/cmeps/med_test_comps/xatm/cime_config/buildnml ./components/cmeps/med_test_comps/xocn/cime_config/buildnml ./components/cice/cime_config/buildnml if os.path.isfile(user_definition): expect(os.path.isfile(file_), "Namelist XML file %s not found!" % file_) if (os.path.isfile(os.path.join(rundir,rpointer)) and (not os.path.isfile(os.path.join(rundir,rpointer + inst_string)))): expect(os.path.isfile(user_nl_file), ./components/cism/cime_config/buildnml expect(os.path.isfile(user_nl_file), if (os.path.isfile(os.path.join(rundir,rpointer)) and (not os.path.isfile(os.path.join(rundir,rpointer + inst_string)))): expect(os.path.isfile(user_nl_file), if os.path.isfile(user_definition): expect(os.path.isfile(file_), "Namelist XML file {} not found!".format(file_)) ./cime/CIME/non_py/src/components/stub_comps_nuopc/sglc/cime_config/buildnml ./cime/CIME/non_py/src/components/stub_comps_nuopc/srof/cime_config/buildnml ./cime/CIME/non_py/src/components/stub_comps_nuopc/siac/cime_config/buildnml ./cime/CIME/non_py/src/components/stub_comps_nuopc/sesp/cime_config/buildnml ./cime/CIME/non_py/src/components/stub_comps_nuopc/swav/cime_config/buildnml ./cime/CIME/non_py/src/components/stub_comps_nuopc/sice/cime_config/buildnml ./cime/CIME/non_py/src/components/stub_comps_nuopc/slnd/cime_config/buildnml ./cime/CIME/non_py/src/components/stub_comps_nuopc/socn/cime_config/buildnml ./cime/CIME/non_py/src/components/stub_comps_nuopc/satm/cime_config/buildnml erik/cesm_alpha2> find . -name buildnml -print -exec grep return {} \; ./components/mizuRoute/cime_config/buildnml ./components/mom/cime_config/buildnml CPL_I2O_PER_CAT_curr = case.get_value("CPL_I2O_PER_CAT") # returns python logical, not string return ./components/rtm/cime_config/buildnml ./components/ww3/cime_config/buildnml ./components/clm/cime_config/buildnml ./components/mosart/cime_config/buildnml ./components/cdeps/dwav/cime_config/buildnml return user_nl_list ./components/cdeps/dglc/cime_config/buildnml return user_nl_list ./components/cdeps/dlnd/cime_config/buildnml return user_nl_list ./components/cdeps/docn/cime_config/buildnml return user_nl_list ./components/cdeps/drof/cime_config/buildnml return user_nl_list ./components/cdeps/dice/cime_config/buildnml return user_nl_list ./components/cdeps/datm/cime_config/buildnml Put available data in datavaliddate list and return it return datavaliddate return None return available_neon_data return user_nl_list ./components/cam/cime_config/buildnml return "{}.cam{}.{}.{}-{}.nc".format( \ # In python3 integer division returns a float value. ./components/cmeps/cime_config/buildnml return libs ./components/cmeps/med_test_comps/xice/cime_config/buildnml ./components/cmeps/med_test_comps/xlnd/cime_config/buildnml ./components/cmeps/med_test_comps/xrof/cime_config/buildnml ./components/cmeps/med_test_comps/xwav/cime_config/buildnml ./components/cmeps/med_test_comps/xglc/cime_config/buildnml ./components/cmeps/med_test_comps/xatm/cime_config/buildnml ./components/cmeps/med_test_comps/xocn/cime_config/buildnml ./components/cice/cime_config/buildnml ./components/cism/cime_config/buildnml return user_nl_list return nmlgen return case.get_value(xmlvar) return case.get_value(xmlvar) return [icesheet_name for icesheet_name in _ICESHEET_OPTIONS return glc_grid.split(GRID_SEP) return "{}.{}.config".format(base, icesheet) return run_type return num_errors """Returns True if the given value is some form of true; otherwise returns False return value return True return False return not _value_is_true(value) ./cime/CIME/non_py/src/components/stub_comps_nuopc/sglc/cime_config/buildnml ./cime/CIME/non_py/src/components/stub_comps_nuopc/srof/cime_config/buildnml ./cime/CIME/non_py/src/components/stub_comps_nuopc/siac/cime_config/buildnml ./cime/CIME/non_py/src/components/stub_comps_nuopc/sesp/cime_config/buildnml ./cime/CIME/non_py/src/components/stub_comps_nuopc/swav/cime_config/buildnml ./cime/CIME/non_py/src/components/stub_comps_nuopc/sice/cime_config/buildnml ./cime/CIME/non_py/src/components/stub_comps_nuopc/slnd/cime_config/buildnml ./cime/CIME/non_py/src/components/stub_comps_nuopc/socn/cime_config/buildnml ./cime/CIME/non_py/src/components/stub_comps_nuopc/satm/cime_config/buildnml

OK, I removed the expect in 18c23f1

…ing two expect statements into one

From review for consistency. Co-authored-by: Bill Sacks <sacks@ucar.edu>

Another review suggestion for consistency in RE. Co-authored-by: Bill Sacks <sacks@ucar.edu>

From review make RE more consistent. Co-authored-by: Bill Sacks <sacks@ucar.edu>

RE consistency Co-authored-by: Bill Sacks <sacks@ucar.edu>

RE consistency. Co-authored-by: Bill Sacks <sacks@ucar.edu>

…ommented section of XML

…1850_aero_ndep_ozone

…exists as that's covered in check_input_data, add some comments around the check for the DATM_CPLHIST_DIR and also show the variable unresolved so it shows what variables it's built with

…converted, but the code assumes because they are only divided into 2 fields that they are and wrongly divideds by a 1000

mvertens · 2026-08-27T14:36:36Z

    <type>char</type>
    <valid_values></valid_values>
-    <default_value>UNSET</default_value>
+    <default_value>$CIME_OUTPUT_ROOT/archive/$DATM_CPLHIST_CASE/cpl/hist</default_value>


@billsacks - I agree with your statements here. CPLHIST is normally very experiment specific and having the default value be UNST is a more robust way to have users be aware that they need to set this.

mvertens · 2026-08-27T14:42:37Z

    <default_value>UNSET</default_value>
+    <values match="last">
+      <value compset="1850.*_DATM%CPLHIST.*_CLM60">b.e30_alpha08o.B1850C_MTso.ne30_t232_wgx3.330</value>
+    </values>


@ekluzek @billsacks - This will break things for NorESM - since the same pattern match applies.
I am wondering if we could trigger off of the xml variable COUPLING_MODE. This could be queries in buildnml - and then a config setting here could be user -

<value coupling_mode="cesm" compset="1850.*_DATM%CPLHIST.*_CLM60">b.e30_alpha08o.B1850C_MTso.ne30_t232_wgx3.330</value>

The COUPLING_MODE variable is already being used in CMEPS - but should be available for all component combinations I believe.

Perfect. Thanks for pointing that out @mvertens! It is available in my cases, so I'll be able to use it.

I'll add this in and make sure it works. I'll post the commit that fixes once I have it.

Adding coupling_mode to the namelist parsing would have been cake. It proved a little more tricky for adding it to config_component.xml and required a CIME change to do it. I did get something that I think is correct, but I couldn't test it with just using a testmod that sets COUPLING_MODE==noresm. So I'll figure out a unit test for it in CIME.

Here's the PR in CIME to do that...

ESMCI/cime#5044

Please look that over and give suggestions there. I think this is a reasonable thing to do for cime_mode==cesm. But, it does only do it for CESM and not all models. That's probably good for limiting the impact, but it's also good to minimize places where cime_model has to be checked. Also note that this could be done out of the box without a CIME update if CIME_MODEL were different between CESM and NorESM. So I wonder if using CIME_MODEL to distringuish between CESM and NorESM isn't the better long term solution. But, this seems like it should be OK for now.

Adding coupling_mode to the namelist parsing would have been cake. It proved a little more tricky for adding it to config_component.xml and required a CIME change to do it. I did get something that I think is correct, but I couldn't test it with just using a testmod that sets COUPLING_MODE==noresm. So I'll figure out a unit test for it in CIME.

Here's the PR in CIME to do that...

ESMCI/cime#5044

Please look that over and give suggestions there. I think this is a reasonable thing to do for cime_mode==cesm. But, it does only do it for CESM and not all models. That's probably good for limiting the impact, but it's also good to minimize places where cime_model has to be checked. Also note that this could be done out of the box without a CIME update if CIME_MODEL were different between CESM and NorESM. So I wonder if using CIME_MODEL to distringuish between CESM and NorESM isn't the better long term solution. But, this seems like it should be OK for now.

@ekluzek - thanks for taking this on. I did not appreciating the difficulty of doing this in config_component.xml. I totally agree that the right solution is to use CIME_MODE - but that is a much more difficult to implement - particularly given the numerous places in CIME that currently need to be touched. I am happy to test this for COUPLING_MODE=noresm with your changes in CIME. But also - given the time crunch for CESM3 - if you want to defer this and bring in your changes to CDEPS without the changes to CIME - I would understand that.

mvertens · 2026-08-27T14:43:37Z

      <value compset="1850.*_DATM%CRUJRA2024[b]*">1</value>
      <value compset="1850.*_DATM%CRUv7">1</value>
      <value compset="1850.*_DATM%GSW">1</value>
+      <value compset="1850.*_DATM%CPLHIST.*_CLM60">1</value>


Since the default value is already 1 - why are all of the other combinations being queries here. The only values that should be listed in the match here are those that are not 1.

That is a really good point, and you are exactly right. There is no reason to set something to the already default value.

mvertens · 2026-08-27T14:44:52Z

      <value compset="1850.*_DATM%CRUv7">1901</value>
      <value compset="1850.*_DATM%GSW">1901</value>
      <value compset="1850.*_DATM%NLDAS2">0</value>  <!-- Unsupported -->
+      <value compset="1850.*_DATM%CPLHIST.*_CLM60">101</value>


Why is the setting 101? Don't we want to align 1850 data with year 1? I think I am missing something here.

This is for the years of data that you want to run over from the CPLHIST data for your case. Most likely that data won't be for year one, as you might skip to later spinup years and skip the initial spinup years for the B case.

In this case for the preliminary data I only have year 101, so it needs to point to the data years that are available.

mvertens · 2026-08-27T14:45:14Z

      <value   compset="1850.*_DATM%CRUv7">1920</value>
      <value   compset="1850.*_DATM%GSW">1920</value>
      <value   compset="1850.*_DATM%NLDAS2">-1</value>  <!-- Unsupported -->
+      <value   compset="1850.*_DATM%CPLHIST.*_CLM60">101</value>


Same question about year 101 here.

mvertens · 2026-08-27T14:51:08Z

+      <var>atmImp_Faxa_ndep1  Faxa_ndep_nhx</var>
+      <var>atmImp_Faxa_ndep2  Faxa_ndep_noy</var>


I have several questions:

when did the filenames_advance get added - and what does it do?

what is the solution to Read in the units for variables on the stream files #436? Is there new code to be added to check the units? Will that be part of this PR?

…NDEP

…t the comment for clarity on that same line

@mvertens

…nflict with noresm, set DATM_CPLHIST_DIR to UNSET as requested by @mvertens and @billsacks, but add to the description about how to set it using \$DATM_CPLHIST_CASE and gives and example path to use in your archive directory for a previous case you ran, remove setting of year align to 1 for CESM CPLHIST, and remove dependency on CLM version and make it just _CLM for these CPLHIST settings we are only supporting modern CPLHIST files, so if you did CPLHIST with running CLM50 you`d want it to work the same in those cases as well

billsacks · 2026-09-02T17:44:19Z

I haven't looked at all of the recent comments here (I've been waiting for a go-ahead from @ekluzek and then figure I'll look at everything at once), but wanted to share a thought, motivated by looking at ESMCI/cime#5044

As I noted in that PR, I think that depending on coupling_mode here is problematic. Long-term I know we want to add a cime_model differentiator between cesm and noresm, but that isn't feasible in the near-term.

Instead, I wondered if it would make sense to be more specific in the compset long name rather than just having DATM%CPLHIST. I feel like, even just considering CESM, we could want different CPLHIST forcing datasets. And this becomes especially true when considering both CESM and NorESM.

For example, could we have something like DATM%CPLHIST-CESM3SPINUP?

I also recognize that this PR is time critical. So I'm okay with it coming in in an imperfect state and then iterating on this in a following PR if there isn't time to get it totally right initially.

billsacks · 2026-09-02T18:00:59Z

I just talked to @mvertens . She likes the idea of differentiating based on compset and feels that's the right way to do it. She would prefer DATM%CPLHIST%CESM3SPINUP (i.e., multiple modifiers). I'm fine either way. We both feel that we can go with whichever is most straightforward to implement - DATM%CPLHIST-CESM3SPINUP or DATM%CPLHIST%CESM3SPINUP. She also agrees that this can come in in a not-yet-complete state and we can iterate on it, so the land spinup can start on time.

ekluzek · 2026-09-02T20:33:29Z

      <value compset="^OMIP_DATM%IAF.*_POP2%[^_]*ECO">omip.iaf</value>
      <value compset="^OMIP_DATM%JRA.*_POP2%[^_]*ECO">omip.jra</value>
-      <value compset="_DATM%CPLHIST">cplhist</value>
+      <value coupling_mode="cesm" compset="_DATM%CPLHIST_CLM">cplhist</value>


@klindsay28 @mnlevy1981 and I all like dropping the _CLM and going back to the original (for CESM). So this would go back to the original.

ekluzek · 2026-09-02T20:42:06Z

+          NOTE: This needs to be coordinated with CCSM_BGC in CMEPS, and with either the ocean or land model
+            For CLM coordinate it with the CLM_CO2_TYPE variable
+            For MOM coordinate it with the OCN_CO2_TYPE variable
+    </desc>


@klindsay28 has a brilliant idea, that we probably don't really need CCSM_BGC anymore because NUOPC now won't pass CO2 related fields unless some subcomponent needs it. He checked CMEPS and it looks like it's ignored.

I'll will check this out, as this might really simplify things.

@klindsay28 - as I just mentioned to @ekluzek - Based on my quick look through the code, it looks to me like CCSM_BGC is still used to set a few CMEPS variables: CCSM_BGC sets BGC_MODE in buildnml, which in turn sets the defaults for flds_co2a, flds_co2b, flds_co2c, and flds_bgc_oi. Those aren't used in CMEPS itself, but are used in CTSM (and maybe other components?) as the mechanism to figure out what the co2 coupling is. So it looks like you're right that CMEPS doesn't need this directly, presumably because it does exactly what you're suggesting, but this is still used to coordinate the co2 fields in at least some components.

ekluzek · 2026-09-02T20:47:43Z

    <valid_values></valid_values>
    <default_value>UNSET</default_value>
+    <values match="last">
+      <value coupling_mode="cesm" compset="1850.*_DATM%CPLHIST.*_CLM">$DIN_LOC_ROOT/cplhist/cesm3_0/$DATM_CPLHIST_CASE/atm</value>


@klindsay28 and @mnlevy1981 suggest that we use "cpl/hist" at the end here rather than "atm" to preserve the archive directory structure.

ekluzek · 2026-09-02T20:53:45Z

      <value compset="1850.*_DATM%CRUv7">1901</value>
      <value compset="1850.*_DATM%GSW">1901</value>
      <value compset="1850.*_DATM%NLDAS2">0</value>  <!-- Unsupported -->
+      <value coupling_mode="cesm" compset="1850.*_DATM%CPLHIST.*_CLM">101</value>


Add a line for MOM6 with the same year, as later MOM6 will have different years.

ekluzek · 2026-09-02T20:58:35Z

+      <meshfile>$DIN_LOC_ROOT/share/meshes/fv0.9x1.25_141008_polemod_ESMFmesh.nc</meshfile>
+    </stream_meshfile>
+    <stream_datafiles>
+      <file>$DIN_LOC_ROOT/atm/cam/chem/trop_strat_mam5_ts4_aero_spinup/ndep/fndep_clm_B1850C_MT4s.1850monthly.ne30_c251220.nc</file>


There's a new file to update to. I might as well use the very latest one here though.

We know this will be changing, and can update as new ones are made or we get the final version.

ekluzek added 6 commits April 4, 2026 14:05

Set defaults for DATM_CPLHIST_* settings for 1850 with DATM coupled t…

d9f75e9

…o CLM

Make the default setting for DATM_CPLHIST_DIR under the users archive…

67a8f18

… directory location for the DATM_CPLHIST_CASE under cpl/hist subdirectory

Correct the filenames for cplhist mode for PRES_NDEP

fcb28ad

Add clim_1850_cesm3_for_cmip7

a72518b

Add clim_1850_cesm3_for_cmip7

b6ebc5c

CPLHIST settings for CLM should be just CLM60, to preserve older answers

ab6e2d7

ekluzek added enhancement New feature or request CESM Only labels Apr 15, 2026

ekluzek added this to LMWG: Sprint Planning Board Apr 15, 2026

ekluzek added answers are bfb Responsibility: CTSM Responsibility to manage and accomplish this issue is the CTSM Software group labels Apr 15, 2026

github-project-automation Bot moved this to Todo in LMWG: Sprint Planning Board Apr 15, 2026

Merge tag 'cdeps1.0.94' into 1850_aero_ndep_ozone

82e13db

cdeps1.0.94

ekluzek mentioned this pull request May 1, 2026

Create new auxiliary history file for cplhist test with ndep data ESCOMP/CTSM#1844

Open

ekluzek self-assigned this May 11, 2026

ekluzek moved this from Todo to In Progress in LMWG: Sprint Planning Board May 11, 2026

Merge branch 'main' into 1850_aero_ndep_ozone

acebf06

ekluzek changed the title ~~1850 aero ndep ozone~~ 1850 CPLHIST aero ndep ozone Jul 14, 2026

ekluzek commented Aug 18, 2026

View reviewed changes

Comment thread datm/cime_config/config_component.xml Outdated

ekluzek changed the title ~~1850 CPLHIST aero ndep ozone~~ 1850 CPLHIST ndep Aug 18, 2026

ekluzek added 7 commits August 18, 2026 13:54

Merge tag 'cdeps1.0.105' into 1850_aero_ndep_ozone

c22f68b

cdeps1.0.105 Conflicts: datm/cime_config/config_component.xml

Needed to get the right variables for ndep in cplhist mode

07b976f

Spin off some functions to modularize the code, to make way for addin…

2e0473f

…g more

Add headers to new functions, and do suggestions from co-pilots revie…

6fb38a9

…w including important points about clm_usr_dat_name and available_neon_data

Do error checking for CPLHIST mode, this resolves ESCOMP#399

2523ab1

Let 1850 cases with CLM60 use cmip7_20tr for CO2 timeseries

3f51f39

Missing for PTS_LAT/LON was actually -999.99 so set it to a variable …

7842d5a

…and use it to define it, and also check that if one is set the other is as well, and then ensure single_column is set at the end of the subroutine

ekluzek added 2 commits August 25, 2026 11:33

Remove cplhist testmod as realized in code review

ec8c0a1

Set YR_ALIGN for CPLHIST to 1, as the normal thing is to start with y…

8106de7

…ear 1 for CPLHIST spinup cases

ekluzek changed the title ~~1850 CPLHIST ndep~~ Changes to CPLHIST mode for robustness and getting the right default settings for CLM and an updated 1850 ndep file Aug 25, 2026

Fix XML syntax error

816b01f

billsacks requested changes Aug 25, 2026

View reviewed changes

ekluzek and others added 11 commits August 26, 2026 11:41

Simple changes from code review adding more to docstrings, and combin…

967ef1f

…ing two expect statements into one

Update datm/cime_config/config_component.xml

339bf37

From review for consistency. Co-authored-by: Bill Sacks <sacks@ucar.edu>

Update datm/cime_config/config_component.xml

458df47

Another review suggestion for consistency in RE. Co-authored-by: Bill Sacks <sacks@ucar.edu>

Update datm/cime_config/config_component.xml

56a9e5e

From review make RE more consistent. Co-authored-by: Bill Sacks <sacks@ucar.edu>

Update datm/cime_config/config_component.xml

4d0489a

RE consistency Co-authored-by: Bill Sacks <sacks@ucar.edu>

Update datm/cime_config/config_component.xml

1083247

RE consistency. Co-authored-by: Bill Sacks <sacks@ucar.edu>

A couple things caught in the code review, fix a typo, and remove a c…

7d61189

…ommented section of XML

Merge branch '1850_aero_ndep_ozone' of github.com:ekluzek/CDEPS into …

2951f37

…1850_aero_ndep_ozone

More things from review, remove checking if DATM_CPLHIST_DOMAIN_FILE …

18c23f1

…exists as that's covered in check_input_data, add some comments around the check for the DATM_CPLHIST_DIR and also show the variable unresolved so it shows what variables it's built with

Fix expect so that PLUMBER2 sites work

07846d8

Add a note on how ndep for cplhist is in units that don't need to be …

baddde6

…converted, but the code assumes because they are only divided into 2 fields that they are and wrongly divideds by a 1000

mvertens requested changes Aug 27, 2026

View reviewed changes

ekluzek added 3 commits August 27, 2026 11:59

Add a comment about the clim_1850_cesm3_for_cmip7 option to DATM_PRES…

93d3b15

…NDEP

Remove accidental character that screwed up the XML syntax, and adjus…

1b961f0

…t the comment for clarity on that same line

ekluzek mentioned this pull request Sep 2, 2026

add stream field specific unit conversions in stream_defintion_MODEL.xml #438

Open

ekluzek commented Sep 2, 2026

View reviewed changes

		<var>atmImp_Faxa_ndep1 Faxa_ndep_nhx</var>
		<var>atmImp_Faxa_ndep2 Faxa_ndep_noy</var>

		expect(cplhist_case is not None, "DATM_CPLHIST_CASE must be set when using cplhist mode")
		expect(cplhist_case != "UNSET", "DATM_CPLHIST_CASE must be set when using cplhist mode")

Uh oh!

Conversation

ekluzek commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of changes

Specific notes

Uh oh!

wwieder commented May 20, 2026

Uh oh!

billsacks commented May 20, 2026

Uh oh!

wwieder commented May 21, 2026

Uh oh!

Uh oh!

ekluzek commented Aug 25, 2026

Uh oh!

ekluzek commented Aug 25, 2026

Uh oh!

billsacks left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

ekluzek commented Apr 15, 2026 •

edited

Loading

mvertens Aug 29, 2026 •

edited

Loading