feat(compare_namelists): add TOML support for compare_namelists and mizuRoute baseline comparison

[johnpaulalex]

Description of changes

Adds native TOML configuration comparison support to CIME/compare_namelists.py (_compare_tomls) and generalizes _do_full_nl_comp in CIME/case/case_cmpgen_namelists.py to handle format transitions (e.g. .control <-> .toml) generically across all CIME components without component-specific string hardcoding.

Adds dependency fallback install_requires=["tomli; python_version < '3.11'"] in setup.py for Python versions below 3.11.

Expected transition order for mizuRoute going to TOML format:

• 1. old baselines are all control
• 2. new cases write both toml & control; when they compare with baseline, they continue to use control
• 3. we transition baselines to toml; when cases run against those newer baselines, they use toml
• 4. Eventually all baselines are toml

Specific notes

Contributors other than yourself, if any:

• @johnpaulalex

Linked issues addressed, if any:

• None

Description of generative AI usage:

• Google Antigravity was used to write the code and tests, followed by human-guided verification.

Answer Changes & Scientific Impact

• Bit-for-Bit (B4B) with baseline master
• Roundoff-level differences only
• Expected Answer Changes (ECA)

User Interface & Namelist Changes

• Namelist / Defaults modified? No
• XML / Build script changes? No

Testing planned or performed, if any:

• Executed unit tests in CIME/tests/test_unit_compare_namelists.py and CIME/tests/test_unit_cmpgen_namelists_mizuroute.py (6/6 passing).

PR branch hash-tag: bc8143e5066e409dbebcda08dcf2f520ea85c2b0

[ekluzek]

Adding support for toml files to store namelist file type content is a nice addition. And this can be done before it's done in mizuRoute, it's just adding additional capability.

Largely this is fine, but I'd like to see it a little more general so that mizuRoute doesn't have to be in the filenames.

[ekluzek]

We talked about a limitation of the standard tomllib library used here is that it removes comments. Which isn't great, but n this context for a comparison it's probably fine.

[ekluzek]

This should be fine, since python 3.9 is already at end of life. And 3.11 is only having security updates until 2028. So shouldn't be that contraversial to update to 3.11 from 3.9.

There's also a python version checker in

CIME/core/config/bootstrap.py

and that perhaps is where this should be set. But, there's a few different places this is done, so I'll defer to someone else who can suggest where this should be done.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 37.66%. Comparing base (f8e59ff) to head (bc8143e).
⚠️ Report is 115 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #5024      +/-   ##
==========================================
+ Coverage   28.48%   37.66%   +9.18%     
==========================================
  Files         262      134     -128     
  Lines       38490    19655   -18835     
  Branches     8146     4166    -3980     
==========================================
- Hits        10964     7404    -3560     
+ Misses      26268    11390   -14878     
+ Partials     1258      861     -397     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ekluzek]

@johnpaulalex I updated the header to outline the transition for mizuRoute. The conversion from control file format to TOML format is unique to mizuRoute. Although there is a desire to move all CESM code to use a standard text file format that's independent of compiler. So we do want to remove namelist handling, but that will be way down the road when each CESM/E3SM/NorESM subcomponents have all transitioned away from namelist. I expect that to be years down the road.

Note that the control file format that mizuRoute uses is a specific home grown file format not used anywhere else. So moving away from it is a higher priority for mizuRoute than for other components.