Skip to content

Additional fix for threading and pe-layout NBFB issues in UCI-chem codes - #70

Merged
wlin7 merged 1 commit into
NGD_v3atmfrom
wlin/atm/fix_pet_pem_nbfb2
May 4, 2023
Merged

Additional fix for threading and pe-layout NBFB issues in UCI-chem codes#70
wlin7 merged 1 commit into
NGD_v3atmfrom
wlin/atm/fix_pet_pem_nbfb2

Conversation

@wlin7

@wlin7 wlin7 commented May 4, 2023

Copy link
Copy Markdown
Collaborator

. Fixed array shape pdeldry in lin_strat_chem.F90 that would failed debug mode
. Fixed codes for surface emission section in mo_gas_phase_chemdr.F90 that
cause round-off NBFB when threading or pe-layout changes

[NBFB]

. Fixed array shape pdeldry in lin_strat_chem.F90 that would failed debug mode
. Fixed codes for surface emission section in mo_gas_phase_chemdr.F90 that
   cause round-off NBFB when threading or pe-layout changes

[NBFB]
@wlin7
wlin7 requested review from jinboxie and keziming May 4, 2023 05:26
@wlin7

wlin7 commented May 4, 2023

Copy link
Copy Markdown
Collaborator Author

Sorry, @keziming , @jinboxie , please use this PR instead. #69 was created without including #62 merged by @jinbo, and there was some conflict. This is a clean one.

@wlin7 wlin7 added bug fix PR non-BFB PR makes roundoff changes to answers labels May 4, 2023
@jinboxie

jinboxie commented May 4, 2023

Copy link
Copy Markdown

Hi Wuyin, I was doing the modification of the xsfc in my new code in #62 for NGD_v3atm
xsfc(1,:)= linoz_o3_clim_srf(:,1) ! ozone surface constant (varying in lat)
xsfc(2,:)= linoz_n2o_clim_srf(:,1) ! n2o (constant throughout latitude)
xsfc(3,:)= linoz_noy_clim_srf(:,1) !noylnz
xsfc(4,:)= linoz_ch4_clim_srf(:,1)
like that of E3SM-Project/E3SM#5661. Would it face the same problem?

The error in the block starting at line 367 can be fixed with explicit upper bound

   xsfc(1,:)=   linoz_o3lbs(:ncol,pver) !  ozone surface constant (varying in lat)
   xsfc(2,:)=   linoz_n2o_clim(:ncol,pver) !  n2o (constant throughout latitude)
   xsfc(3,:)=   linoz_o3lbs(:ncol,pver)*3.e-3_r8  !noylnz
   xsfc(4,:)=   linoz_ch4_clim(:ncol,pver) ! ch4 (constant throughout latitude)

Then there is more starting from line 390, also with array from the 'fields' data structure. Again, we really need to know why some of the chunk having mismatched 1st dimension. (ncol sees both 14 and 15, mostly has matching data dimension from fields. But why sometimes ncol=14, data from fields is 15, while PCOLS=16?)

@wlin7

wlin7 commented May 4, 2023

Copy link
Copy Markdown
Collaborator Author

Could face the same problem, @jinboxie . The 1st dimension size for data array in the fields structure is sometimes puzzling. Using explicit upper bound of ncol for the 1st dimension should fix the runtime error. Just need to make sure the data in fields are correct. You must be more familiar with the fields data structure. Are you able to track how the array is allocated for the elements in fields. I assume it always uses chunk specific ncol for the 1st dimension, but I haven't located it in the codes.

@wlin7

wlin7 commented May 4, 2023

Copy link
Copy Markdown
Collaborator Author

Merged to NGD_v3atm.

@jinboxie

jinboxie commented May 4, 2023

Copy link
Copy Markdown

Could face the same problem, @jinboxie . The 1st dimension size for data array in the fields structure is sometimes puzzling. Using explicit upper bound of ncol for the 1st dimension should fix the runtime error. Just need to make sure the data in fields are correct. You must be more familiar with the fields data structure. Are you able to track how the array is allocated for the elements in fields. I assume it always uses chunk specific ncol for the 1st dimension, but I haven't located it in the codes.

Yes Wuyin @wlin7 , there are some thoughts on the bounds issue here.
E3SM-Project/E3SM#5661
Also, for the fields, they are first set in the trcdata_init in components/eam/src/chemistry/utils/tracers.F90. Depending on the data is vertical or surface variables. Take the linoz_o3_clim as an example, it is set as (pcols, pver, begchunk:endchunk) or (pcols, 1, begchunk:endchunk).

if ( .not. file%in_pbuf(f) ) then
          if ( flds(f)%srf_fld .or. file%top_bndry ) then
             allocate( flds(f)         %data(pcols,1,begchunk:endchunk), stat=astat   )
          else
             allocate( flds(f)         %data(pcols,pver,begchunk:endchunk), stat=astat   )
          endif
          if( astat/= 0 ) then
             write(iulog,*) 'trcdata_init: failed to allocate flds(f)%data array; error = ',astat
             call endrun
          end if
       else
          flds(f)%pbuf_ndx = pbuf_get_index(flds(f)%fldnam,errcode)
       endif

And then get output from loc_arr from read_za_trcdata in components/eam/src/chemistry/utils/tracers.F90.
Within read_za_trcdata, we get

call read_za_trc( fids(i), flds(f)%var_id, flds(f)%input(i)%data, strt3, cnt3, file, &
(/ flds(f)%order(ZA_LATDIM),flds(f)%order(ZA_LEVDIM) /) )

subroutine read_za_trc( fid, vid, loc_arr, strt, cnt, file, order )


do c=begchunk,endchunk
       ncols = get_ncols_p(c)
       call get_rlat_all_p(c, pcols, to_lats)
       call lininterp_init(file%lats, file%nlat, to_lats, ncols, 1, lat_wgts)
       do k=1,file%nlev
          call lininterp(wrk2d_in(:,k), file%nlat, wrk(1:ncols), ncols, lat_wgts)
          loc_arr(1:ncols,k,c-begchunk+1) = wrk(1:ncols)
       end do
       call lininterp_finish(lat_wgts)
    end do

loc_arr is output to the flds(f)%input(i)%data allocated above.
This code would output the loc_arr that passes to the field and hence the linoz_o3_clim data in components/eam/src/chemistry/mozart/lin_strat_chem.F90

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug fix PR non-BFB PR makes roundoff changes to answers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants