Skip to content

add stream field specific unit conversions in stream_defintion_MODEL.xml #438

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mvertens wants to merge 3 commits into
base: main
Choose a base branch
from
+139 −38
Open

add stream field specific unit conversions in stream_defintion_MODEL.xml #438

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
8cae291
Add optional stream_scale_factor for unit conversion on read and fix …
Sep 1, 2026
afc7bb7
fixed specification of forcing data for presndep.cplhist
Sep 1, 2026
63b8e90
changes made to address issues raised in PR review in ESCOMP
Sep 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 23 additions & 18 deletions datm/cime_config/stream_definition_datm.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4933,8 +4933,8 @@
</stream_datafiles>
<stream_datavars>
<!-- the following stream fields are in units of g/m2/sec - need conversion to kg/m2/sec-->
<var>NDEP_NHx_month Faxa_ndep_nhx</var>
<var>NDEP_NOy_month Faxa_ndep_noy</var>
<var>NDEP_NHx_month Faxa_ndep_nhx 1.0d-3</var>
<var>NDEP_NOy_month Faxa_ndep_noy 1.0d-3</var>
</stream_datavars>
<stream_lev_dimname>null</stream_lev_dimname>
<stream_mapalgo>
Expand Down Expand Up @@ -4966,8 +4966,8 @@
</stream_datafiles>
<stream_datavars>
<!-- the following stream fields are in units of g/m2/sec - need conversion to kg/m2/sec-->
<var>NDEP_NHx_month Faxa_ndep_nhx</var>
<var>NDEP_NOy_month Faxa_ndep_noy</var>
<var>NDEP_NHx_month Faxa_ndep_nhx 1.0d-3</var>
<var>NDEP_NOy_month Faxa_ndep_noy 1.0d-3</var>
</stream_datavars>
<stream_lev_dimname>null</stream_lev_dimname>
<stream_mapalgo>
Expand Down Expand Up @@ -4998,8 +4998,9 @@
<file>$DIN_LOC_ROOT/lnd/clm2/ndepdata/fndep_clm_hist_b.e21.BWHIST.f09_g17.CMIP6-historical-WACCM.ensmean_1849-2015_monthly_0.9x1.25_c180926.nc</file>
</stream_datafiles>
<stream_datavars>
<var>NDEP_NHx_month Faxa_ndep_nhx</var>
<var>NDEP_NOy_month Faxa_ndep_noy</var>
<!-- the following stream fields are in units of g/m2/sec - need conversion to kg/m2/sec-->
<var>NDEP_NHx_month Faxa_ndep_nhx 1.0d-3</var>
<var>NDEP_NOy_month Faxa_ndep_noy 1.0d-3</var>
</stream_datavars>
<stream_lev_dimname>null</stream_lev_dimname>
<stream_mapalgo>
Expand Down Expand Up @@ -5031,8 +5032,8 @@
</stream_datafiles>
<stream_datavars>
<!-- the following stream fields are in units of g/m2/sec - need conversion to kg/m2/sec-->
<var>NDEP_NHx_month Faxa_ndep_nhx</var>
<var>NDEP_NOy_month Faxa_ndep_noy</var>
<var>NDEP_NHx_month Faxa_ndep_nhx 1.0d-3</var>
<var>NDEP_NOy_month Faxa_ndep_noy 1.0d-3</var>
</stream_datavars>
<stream_lev_dimname>null</stream_lev_dimname>
<stream_mapalgo>
Expand Down Expand Up @@ -5064,8 +5065,8 @@
</stream_datafiles>
<stream_datavars>
<!-- the following stream fields are in units of g/m2/sec - need conversion to kg/m2/sec-->
<var>NDEP_NHx_month Faxa_ndep_nhx</var>
<var>NDEP_NOy_month Faxa_ndep_noy</var>
<var>NDEP_NHx_month Faxa_ndep_nhx 1.0d-3</var>
<var>NDEP_NOy_month Faxa_ndep_noy 1.0d-3</var>
</stream_datavars>
<stream_lev_dimname>null</stream_lev_dimname>
<stream_mapalgo>
Expand Down Expand Up @@ -5097,8 +5098,8 @@
</stream_datafiles>
<stream_datavars>
<!-- the following stream fields are in units of g/m2/sec - need conversion to kg/m2/sec-->
<var>NDEP_NHx_month Faxa_ndep_nhx</var>
<var>NDEP_NOy_month Faxa_ndep_noy</var>
<var>NDEP_NHx_month Faxa_ndep_nhx 1.0d-3</var>
<var>NDEP_NOy_month Faxa_ndep_noy 1.0d-3</var>
</stream_datavars>
<stream_lev_dimname>null</stream_lev_dimname>
<stream_mapalgo>
Expand Down Expand Up @@ -5130,8 +5131,8 @@
</stream_datafiles>
<stream_datavars>
<!-- the following stream fields are in units of g/m2/sec - need conversion to kg/m2/sec-->
<var>NDEP_NHx_month Faxa_ndep_nhx</var>
<var>NDEP_NOy_month Faxa_ndep_noy</var>
<var>NDEP_NHx_month Faxa_ndep_nhx 1.0d-3</var>
<var>NDEP_NOy_month Faxa_ndep_noy 1.0d-3</var>
</stream_datavars>
<stream_lev_dimname>null</stream_lev_dimname>
<stream_mapalgo>
Expand Down Expand Up @@ -5164,8 +5165,8 @@
</stream_datafiles>
<stream_datavars>
<!-- the following stream fields are in units of g/m2/sec - need conversion to kg/m2/sec-->
<var>NDEP_NHx_month Faxa_ndep_nhx</var>
<var>NDEP_NOy_month Faxa_ndep_noy</var>
<var>NDEP_NHx_month Faxa_ndep_nhx 1.0d-3</var>
<var>NDEP_NOy_month Faxa_ndep_noy 1.0d-3</var>
</stream_datavars>
<stream_lev_dimname>null</stream_lev_dimname>
<stream_mapalgo>
Expand Down Expand Up @@ -5271,10 +5272,14 @@
<meshfile>$ATM_DOMAIN_MESH</meshfile>
</stream_meshfile>
<stream_datafiles>
<file first_year="$DATM_YR_START" last_year="$DATM_YR_END">$DATM_CPLHIST_DIR/$DATM_CPLHIST_CASE.cpl.ha2x1d.%ym.nc</file>
<file first_year="$DATM_YR_START" last_year="$DATM_YR_END">$DATM_CPLHIST_DIR/$DATM_CPLHIST_CASE.cpl.hx.atm.24h.avrg.%ymd.nc</file>
Comment thread
ekluzek marked this conversation as resolved.
</stream_datafiles>
<stream_datavars>
<var>a2x1d_Faxa_ndep Faxa_ndep</var>
<!-- the mediator writes the two ungridded elements of Faxa_ndep as
separate variables, index 1 being NHx and index 2 NOy.
These are already in kgN/m2/s, so no conversion factor is given. -->
<var>atmImp_Faxa_ndep1 Faxa_ndep_nhx</var>
<var>atmImp_Faxa_ndep2 Faxa_ndep_noy</var>
</stream_datavars>
<stream_lev_dimname>null</stream_lev_dimname>
<stream_mapalgo>
Expand Down
30 changes: 13 additions & 17 deletions datm/datm_pres_ndep_mod.F90
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,9 @@ module datm_pres_ndep_mod
real(r8), pointer :: strm_Faxa_ndep_noy_dry(:) => null() ! stream cmip7 ndep data
real(r8), pointer :: strm_Faxa_ndep_noy_wet(:) => null() ! stream cmip7 ndep data

real(r8), pointer :: strm_Faxa_ndep_nhx(:) => null() ! pre-cmip7 ndep data
real(r8), pointer :: strm_Faxa_ndep_noy(:) => null() ! pre-cmip7 ndep data
real(r8), pointer :: strm_Faxa_ndep_nhx(:) => null() ! pre-cmip7 ndep data and cplhist
real(r8), pointer :: strm_Faxa_ndep_noy(:) => null() ! pre-cmip7 ndep data and cplhist

logical :: use_cmip7_ndep

character(len=*), parameter :: u_FILE_u = &
__FILE__
Expand Down Expand Up @@ -76,20 +75,17 @@ subroutine datm_pres_ndep_init_pointers(exportState, sdat, rc)
call shr_strdata_get_stream_pointer(sdat, 'Faxa_ndep_noy_wet', strm_Faxa_ndep_noy_wet, rc)
if (ChkErr(rc,__LINE__,u_FILE_u)) return

! cmip6 ndep forcing
! cmip6 ndep forcing and cplhist
call shr_strdata_get_stream_pointer( sdat, 'Faxa_ndep_nhx', strm_Faxa_ndep_nhx, rc)
if (ChkErr(rc,__LINE__,u_FILE_u)) return
call shr_strdata_get_stream_pointer( sdat, 'Faxa_ndep_noy', strm_Faxa_ndep_noy, rc)
if (ChkErr(rc,__LINE__,u_FILE_u)) return

! determine use_cmip_ndep module variable
if (associated(strm_Faxa_ndep_nhx_dry) .and. associated(strm_Faxa_ndep_nhx_wet) .and. &
associated(strm_Faxa_ndep_noy_dry) .and. associated(strm_Faxa_ndep_noy_wet)) then
use_cmip7_ndep = .true.
else if (associated(strm_Faxa_ndep_nhx) .and. associated(strm_Faxa_ndep_noy)) then
use_cmip7_ndep = .false.
else
call shr_log_error('datm_ndep_advance: ERROR: no associated stream pointers for ndep forcing', rc=rc)
! Check that one of the two supported field sets was found.
if (.not. ((associated(strm_Faxa_ndep_nhx_dry) .and. associated(strm_Faxa_ndep_nhx_wet) .and. &
associated(strm_Faxa_ndep_noy_dry) .and. associated(strm_Faxa_ndep_noy_wet)) .or. &
(associated(strm_Faxa_ndep_nhx) .and. associated(strm_Faxa_ndep_noy)))) then
call shr_log_error('datm_pres_ndep: ERROR: no associated stream pointers for ndep forcing', rc=rc)
return
end if

Expand All @@ -98,14 +94,14 @@ end subroutine datm_pres_ndep_init_pointers
!===============================================================================
subroutine datm_pres_ndep_advance()

if (use_cmip7_ndep) then
! assume data is in kgN/m2/s
! Separate dry and wet deposition fields are summed where the stream provides
! them; otherwise the combined fields are used.
if (associated(strm_Faxa_ndep_nhx_dry)) then
Faxa_ndep(1,:) = strm_Faxa_ndep_nhx_dry(:) + strm_Faxa_ndep_nhx_wet(:)
Faxa_ndep(2,:) = strm_Faxa_ndep_noy_dry(:) + strm_Faxa_ndep_noy_wet(:)
else
! convert ndep flux to units of kgN/m2/s (input is in gN/m2/s)
Faxa_ndep(1,:) = strm_Faxa_ndep_nhx(:) / 1000._r8
Faxa_ndep(2,:) = strm_Faxa_ndep_noy(:) / 1000._r8
Faxa_ndep(1,:) = strm_Faxa_ndep_nhx(:)
Faxa_ndep(2,:) = strm_Faxa_ndep_noy(:)
end if

end subroutine datm_pres_ndep_advance
Expand Down
25 changes: 25 additions & 0 deletions streams/dshr_strdata_mod.F90
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ module dshr_strdata_mod
use shr_string_mod , only : shr_string_listgetname, shr_string_listisvalid, shr_string_listgetnum

use dshr_stream_mod , only : shr_stream_streamtype, shr_stream_getModelFieldList, shr_stream_getStreamFieldList
use dshr_stream_mod , only : shr_stream_getFieldScaleFactors
use dshr_stream_mod , only : shr_stream_taxis_cycle, shr_stream_taxis_extend, shr_stream_findBounds
use dshr_stream_mod , only : shr_stream_getCurrFile, shr_stream_setCurrFile, shr_stream_getMeshFilename
use dshr_stream_mod , only : shr_stream_init_from_inline, shr_stream_init_from_esmfconfig
Expand Down Expand Up @@ -98,6 +99,7 @@ module dshr_strdata_mod
type(ESMF_RouteHandle) :: routehandle ! stream n -> model mesh mapping
character(len=CL), allocatable :: fldlist_stream(:) ! names of stream file fields
character(len=CL), allocatable :: fldlist_model(:) ! names of stream model fields
real(r8), allocatable :: fldlist_scale(:) ! per-field unit conversion applied on read
integer :: stream_nlev ! number of vertical levels in stream
real(r8), allocatable :: stream_vlevs(:) ! values of vertical levels in stream
integer :: stream_lb ! index of the Lowerbound (LB) in fldlist_stream
Expand Down Expand Up @@ -554,6 +556,13 @@ subroutine shr_strdata_init(sdat, model_clock, stream_name, rc)
return
end if
call shr_stream_getStreamFieldList(sdat%stream(ns), sdat%pstrm(ns)%fldlist_stream)
allocate(sdat%pstrm(ns)%fldlist_scale(nvars), stat=istat)
if ( istat /= 0 ) then
call shr_log_error(subName//&
': allocation error for sdat%pstrm('//toString(ns)//')%fldlist_scale with nvars '//toString(nvars), rc=rc)
return
end if
call shr_stream_getFieldScaleFactors(sdat%stream(ns), sdat%pstrm(ns)%fldlist_scale)

! Create field bundles on model mesh
if (sdat%stream(ns)%readmode=='single') then
Expand Down Expand Up @@ -2132,6 +2141,22 @@ subroutine shr_strdata_readstrm(sdat, per_stream, stream, fldbun_data, &

end if

! Apply this field's unit conversion, if the stream definition gave one as
! a third token on its <var> line. Units belong to the field rather than
! the stream, so each field carries its own factor and a stream may mix
! converted and unconverted fields.
!
! Fill values are left alone: they are markers, not measurements. This is
! separate from the CF scale_factor/add_offset unpacking above, which
! undoes on-disk packing and has already been applied by this point.
if (per_stream%fldlist_scale(nf) /= 1.0_r8) then
if (stream_nlev > 1) then
where (dataptr2d /= r8fill) dataptr2d = dataptr2d * per_stream%fldlist_scale(nf)
else
where (dataptr1d /= r8fill) dataptr1d = dataptr1d * per_stream%fldlist_scale(nf)
end if
end if

if(associated(dataptr2d_src) .and. trim(per_stream%fldlist_model(nf)) .eq. uname) then
! save in dataptr2d_src
dataptr2d_src(1,:) = dataptr1d(:)
Expand Down
81 changes: 78 additions & 3 deletions streams/dshr_stream_mod.F90
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,8 @@ module dshr_stream_mod
public :: shr_stream_findBounds ! return lower/upper bounding date info
public :: shr_stream_getMeshFileName ! return stream filename
public :: shr_stream_getModelFieldList ! return model field name list
public :: shr_stream_getStreamFieldList! return stream file field name list
public :: shr_stream_getStreamFieldList ! return stream file field name list
public :: shr_stream_getFieldScaleFactors ! return per-field unit conversion factors
public :: shr_stream_getPrevFileName ! return previous file in sequence
public :: shr_stream_getNextFileName ! return next file in sequence
public :: shr_stream_getNFiles ! get the number of files in a stream
Expand Down Expand Up @@ -99,6 +100,11 @@ module dshr_stream_mod
type shr_stream_data_variable
character(len=CS) :: nameinfile
character(len=CS) :: nameinmodel
! Optional unit conversion, applied to this field as it is read. Given as
! a third token on the <var> line in the stream definition xml; absent
! means 1.0. Units are a property of the field, not of the stream, so a
! stream may mix converted and unconverted fields.
real(r8) :: scale_factor = 1.0_r8
end type shr_stream_data_variable

type shr_stream_streamType
Expand Down Expand Up @@ -350,8 +356,7 @@ subroutine shr_stream_init_from_xml(streamfilename, streamdat, isroot_task, logu
do n = 1, streamdat(i)%nvars
p => item(varlist, n-1)
call extractDataContent(p, tmpstr)
streamdat(i)%varlist(n)%nameinfile = tmpstr(1:index(tmpstr, " "))
streamdat(i)%varlist(n)%nameinmodel = tmpstr(index(trim(tmpstr), " ", .true.)+1:)
call parse_var_entry(tmpstr, streamdat(i)%varlist(n))
enddo

enddo
Expand Down Expand Up @@ -413,6 +418,10 @@ subroutine shr_stream_init_from_xml(streamfilename, streamdat, isroot_task, logu
if (ChkErr(rc,__LINE__,u_FILE_u)) return
call ESMF_VMBroadCast(vm, streamdat(i)%varlist(n)%nameinmodel, CS, 0, rc=rc)
if (ChkErr(rc,__LINE__,u_FILE_u)) return
rtmp(1) = streamdat(i)%varlist(n)%scale_factor
call ESMF_VMBroadCast(vm, rtmp, 1, 0, rc=rc)
if (ChkErr(rc,__LINE__,u_FILE_u)) return
streamdat(i)%varlist(n)%scale_factor = rtmp(1)
enddo
call ESMF_VMBroadCast(vm, streamdat(i)%meshfile, CL, 0, rc=rc)
if (ChkErr(rc,__LINE__,u_FILE_u)) return
Expand Down Expand Up @@ -1613,6 +1622,72 @@ subroutine shr_stream_getModelFieldList(stream, list)

end subroutine shr_stream_getModelFieldList

!===============================================================================
subroutine parse_var_entry(entry, var)

! Parse one <var> line into its two or three whitespace-separated tokens:
!
! <var>drynhx Faxa_ndep_nhx_dry</var>
! <var>NDEP_NHx_month Faxa_ndep_nhx 1.0e-3</var>
!
! The optional third token is a unit-conversion factor applied to this field
! as it is read. Absent leaves the default of 1.0.

! input/output parameters:
character(len=*) ,intent(in) :: entry
type(shr_stream_data_variable) ,intent(inout) :: var

integer :: pos, next, ios
character(len=CX) :: rest, third
character(len=*),parameter :: subName = '(parse_var_entry) '
!-------------------------------------------------------------------------------

rest = adjustl(entry)

pos = index(trim(rest), " ")
if (pos == 0) then
call shr_sys_abort(subName//" stream var entry needs at least two fields: "//trim(entry), &
file=u_FILE_u, line=__LINE__)
end if
var%nameinfile = rest(1:pos-1)

rest = adjustl(rest(pos+1:))
next = index(trim(rest), " ")
if (next == 0) then
! two tokens: no conversion
var%nameinmodel = trim(rest)
var%scale_factor = 1.0_r8
return
end if
var%nameinmodel = rest(1:next-1)

third = adjustl(rest(next+1:))
read(third, *, iostat=ios) var%scale_factor
if (ios /= 0) then
call shr_sys_abort(subName//" could not read scale factor from stream var entry: "//trim(entry), &
file=u_FILE_u, line=__LINE__)
end if

end subroutine parse_var_entry

!===============================================================================
subroutine shr_stream_getFieldScaleFactors(stream, factors)

! Get the per-field unit-conversion factors, in the same order as
! shr_stream_getStreamFieldList and shr_stream_getModelFieldList.

!input/output parameters:
type(shr_stream_streamType) ,intent(in) :: stream
real(r8) ,intent(out) :: factors(:)
!-------------------------------------------------------------------------------
integer :: i

do i=1,stream%nvars
factors(i) = stream%varlist(i)%scale_factor
enddo

end subroutine shr_stream_getFieldScaleFactors

!===============================================================================
subroutine shr_stream_getStreamFieldList(stream, list)

Expand Down
Loading