Add script split multifield ncs

[joshuatorrance]

This script splits multi-field netCDF files produced for ESM1.6 into single-field files.

It is intended to be used on ESM1.6 atmosphere and ice output.

• Code
• Docs
• Tests

[blimlim]

Thanks @joshuatorrance, this is looking great and very thorough.

I've compared the outputs against um2nc single variable(?) outputs from the 89-single-variable-files branch, using a monthly esm1.6 file.

Overall using nccmp, there are almost no differences. Aside from the ones mentioned in the comments, there are differences in the time units formatting:

DIFFER : LENGTHS : ATTRIBUTE : units : VARIABLE : time : 21 <> 27 : VALUES : "days since 0001-01-01" : "days since 0001-01-01 00:00"

(I couldn't figure out how to configure this with xarray, or if it's possible)

and for some unknown reason, latitude_longitude is given a coordinates attribute with splitnc and not um2nc:

DIFFER : VARIABLE "latitude_longitude" IS MISSING ATTRIBUTE WITH NAME "coordinates" IN FILE "um2nc_split/atmosphere/fld_s03i257_aiihca.pa-000101_1mon.nc"
DIFFER : NUMBER OF ATTRIBUTES : VARIABLE : latitude_longitude : 4 <> 3

(this seems pretty minor though)

I'll check with the daily, 3hrly etc atmosphere outputs and also test it out with some ice outputs.

I've added a few small comments of things I noticed but nothing major.

[blimlim]

I've compared the outputs with the latest code to um2nc with single variable/field/... files, using the monthly, daily, 6hrly, 3hrly, and 1hrly outputs from an ESM1.6 run.

According to nccmp, all the files match up to the formatting for the time units, and occasional coordinate attribute which gets added to latitude_longitude. I don't think either of these are too concerning.

I'll test out a sea ice file now!

[blimlim]

I can't see any issues in daily and monthly sea ice files produced by splitnc:

>>>    ice_mon = xr.load_dataset("/g/data/tm70/sw6175/development/esm1p6/test_splitnc/ice_mon/iceh-1monthly-mean_0376-01.nc")
>>>    ice_mon_split = xr.open_mfdataset("/g/data/tm70/sw6175/development/esm1p6/test_splitnc/ice_mon/output/*1monthly-mean*.nc", compat="identical")
>>>    ice_day = xr.load_dataset("/g/data/tm70/sw6175/development/esm1p6/test_splitnc/ice_mon/iceh-1daily-mean_0376-01.nc")
>>    ice_day_split = xr.open_mfdataset("/g/data/tm70/sw6175/development/esm1p6/test_splitnc/ice_mon/output/*1daily-mean*.nc", compat="identical")


>>> ice_day_split.compute().identical(ice_day)
True

>>> ice_mon_split.compute().identical(ice_mon)
True

@anton-seaice I have some ESM1.6 CICE output split into single variable files using the script in this PR in /g/data/tm70/sw6175/development/esm1p6/test_splitnc/ice_mon/output, and the source files in /g/data/tm70/sw6175/development/esm1p6/test_splitnc/ice_mon. Based on the above, everything looks good but if you are happy to take a quick look that would be much appreciated!

[aidanheerdegen]

I've pushed submit on the feedback so far in case I don't get back to it in a timely fashion.

I have not reviewed the main logic as yet, but don't let me hold you up.

[blimlim]

Thanks @joshuatorrance, this is looking good on my end. Just noticed one small thing on a final read through

[blimlim]

Thanks @joshuatorrance for putting this together, everything's looking good to me!

[anton-seaice]

How is this getting deployed? just in /g/data/vk83/apps/ ? or into the payu containerised environment ?

[anton-seaice]

I think there should be something in global attributes about what operation was done to make the file, having the existing metadata is good:

// global attributes:
                :title = "sea ice model output for CICE" ;
                :contents = "Diagnostic and Prognostic Variables" ;
                :source = "Los Alamos Sea Ice Model (CICE) Version 5" ;
                :time_period_freq = "month_1" ;
                :comment = "This year has 366 days" ;
                :comment2 = "File started on model date 03760201" ;
                :history = "This dataset was created on 2026-04-30 at 09:41:28.7" ;
                :io_flavor = "io_netcdf" ;

but an extra attribute linking to the code used for post processing would be good

[anton-seaice]

Is the goal to exactly follow the ocean filename convention?

aice_iceh-1daily-mean_0376-01.nc

should be

ice-2d-aice-1daily-mean_0376-01.nc

right ?

[anton-seaice]

This looks great @joshuatorrance - thanks for all your work on this

[joshuatorrance]

Is the goal to exactly follow the ocean filename convention?

aice_iceh-1daily-mean_0376-01.nc

should be

ice-2d-aice-1daily-mean_0376-01.nc

right ?

Yup, the current naming scheme is just a place holder.

We're intending to mimic the ocean filename patterns. Exactly how that gets wired up is the next challenge - it's likely it'll require more updates to this script ( #4 )

[joshuatorrance]

I think there should be something in global attributes about what operation was done to make the file, having the existing metadata is good:

// global attributes:
                :title = "sea ice model output for CICE" ;
                :contents = "Diagnostic and Prognostic Variables" ;
                :source = "Los Alamos Sea Ice Model (CICE) Version 5" ;
                :time_period_freq = "month_1" ;
                :comment = "This year has 366 days" ;
                :comment2 = "File started on model date 03760201" ;
                :history = "This dataset was created on 2026-04-30 at 09:41:28.7" ;
                :io_flavor = "io_netcdf" ;

but an extra attribute linking to the code used for post processing would be good

Oooh good point! splitnc should update the history field.

[blimlim]

Thanks @joshuatorrance, this looks great!

[blimlim]

Thanks @joshuatorrance!