Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -335,34 +335,67 @@ subroutine init_mesh( config, &
!===========================================================
if (mesh_file(dst) == mesh_file(src)) then
call load_global_mesh( mesh_file(dst), mesh_names )

! Partition the global meshes
!===========================================================
call create_local_mesh( mesh_names(dst:dst), &
local_rank, total_ranks, &
decomposition_dst, &
stencil_depths, &
generate_inner_halos(dst), &
partitioner_dst, &
enforce_constraints = .false. )
call create_local_mesh( mesh_names(src:src), &
local_rank, total_ranks, &
decomposition_src, &
stencil_depths, &
generate_inner_halos(src), &
partitioner_src, &
enforce_constraints = .false. )

! Create the associated local mesh maps
!===========================================================
if (regrid_method == regrid_method_map) then
call create_local_mesh_maps( mesh_file(dst) )
end if

! Clear the
call global_mesh_collection%clear()

else
call load_global_mesh( mesh_file(dst), mesh_names(dst) )

! SOURCE
call load_global_mesh( mesh_file(src), mesh_names(src) )
endif
write( log_scratch_space, '(A)' ) &
'Loaded source mesh'
call log_event(log_scratch_space, log_level_debug)

call create_local_mesh( mesh_names(src:src), &
local_rank, total_ranks, &
decomposition_src, &
stencil_depths, &
generate_inner_halos(src), &
partitioner_src, &
enforce_constraints = .false. )

call global_mesh_collection%clear()

! DESTINATION
call load_global_mesh( mesh_file(dst), mesh_names(dst) )
write( log_scratch_space, '(A)' ) &
'Loaded destination mesh'
call log_event(log_scratch_space, log_level_debug)

call create_local_mesh( mesh_names(dst:dst), &
local_rank, total_ranks, &
decomposition_dst, &
stencil_depths, &
generate_inner_halos(dst), &
partitioner_dst, &
enforce_constraints = .false. )

call global_mesh_collection%clear()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this line is here because the global mesh is not needed by lfric2lfric. Why is it not the case when the source and destination meshes are read from the same file? Shouldn't be this line outside the if sentence?

A comment in the PR indicating that this change is taking place, apart from the addition of the new tests, would be useful.

I could not find an associated issue where the details of the PR are discussed, although I know this is part of larger work.

@cjohnson-pi cjohnson-pi Jul 27, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue is #440 - now also added above.

@cjohnson-pi cjohnson-pi Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've adjusted the code to also clear the global mesh for the case when the source and destination mesh are from the same file.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I meant is that the clear global mesh could be done outside the
if (mesh_file(dst) == mesh_file(src)) then
condition, but it is also possible to do it like you did.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Juan. Sorry I didn't explain this. Yes I did consider putting the clear_global_mesh outside the if as you had suggested. But the reason I didn't is that when the dst is not the same as src, 2 clear_global_meshes are required - one after each create_local_mesh. So it seemed a better code structure to me to add the clear_global_mesh immediately after the create_local_mesh or create_local_maps. i.e. read global, create local_from_global, clear global. Repeat.


! Partition the global meshes
!===========================================================
call create_local_mesh( mesh_names(dst:dst), &
local_rank, total_ranks, &
decomposition_dst, &
stencil_depths, &
generate_inner_halos(dst), &
partitioner_dst, &
enforce_constraints = .false. )

call create_local_mesh( mesh_names(src:src), &
local_rank, total_ranks, &
decomposition_src, &
stencil_depths, &
generate_inner_halos(src), &
partitioner_src, &
enforce_constraints = .false. )

! Read in the global intergrid mesh mappings,
! then create the associated local mesh maps
!===========================================================
if (regrid_method == regrid_method_map) then
call create_local_mesh_maps( mesh_file(dst) )
end if

end if ! prepartitioned
Expand Down
67 changes: 67 additions & 0 deletions rose-stem/app/lfric2lfric/file/iodef_diag_file.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
<!-- iodef file for source files that have originated from lfric_atm output (lfric_diag) files -->
<!-- This uses grid_ref=full_level_face grid instead of domain_ref=checkpoint_Wtheta -->
<?xml version="1.0" encoding="UTF-8"?>
<simulation>

<!-- File definition for read (Source file) -->
<context id = "lfric2lfric_source">

<!-- Include axis and grid definitions -->
<axis_definition src="./axis_def_main.xml"/>
<axis_definition src="./axis_def_urban1t.xml"/>
<grid_definition src="./grid_def_main.xml"/>

<!-- Local file definitions -->
<file_definition type="one_file" par_access="collective" time_counter="none">

<!-- File definition for LFRic diagnostic file read -->
<file id="lfric_checkpoint_read" name="lfric_checkpoint_read" mode="read" output_freq="1ts" convention="UGRID" cyclic="true" enabled=".TRUE.">
<!-- Atmosphere fields -->
<field id="restart_theta" name="theta" long_name="air_potential_temperature" standard_name="air_potential_temperature" unit="K" grid_ref="full_level_face_grid" operation="once" />
<field id="restart_rho" name="rho" long_name="air_density" standard_name="air_density" unit="kg m-3" grid_ref="half_level_face_grid" operation="once" />
</file>

</file_definition>
</context>

<!-- File definition for write (Destination file) -->
<context id = "lfric2lfric_destination">

<!-- Include axis and grid definitions -->
<axis_definition src="./axis_def_main.xml"/>
<axis_definition src="./axis_def_urban1t.xml"/>
<grid_definition src="./grid_def_main.xml"/>

<!-- Local file definitions -->
<file_definition type="one_file" par_access="collective" time_counter="none">

<!-- File definition for LFRic checkpoint -->
<file id="lfric_checkpoint_write" name="lfric_checkpoint_write" mode="write" output_freq="1ts" convention="UGRID" enabled=".TRUE.">

<!-- Atmosphere fields -->
<field id="checkpoint_theta" name="theta" long_name="air_potential_temperature" standard_name="air_potential_temperature" unit="K" prec="8" domain_ref="checkpoint_Wtheta" operation="once" />
<field id="checkpoint_rho" name="rho" long_name="air_density" standard_name="air_density" unit="kg m-3" prec="8" domain_ref="checkpoint_W3" operation="once" />
</file>

</file_definition>

</context>

<context id = "xios">
<variable_definition>
<variable_group id = "buffer">
<variable id = "optimal_buffer_size" type = "string" >performance</variable>
<variable id = "buffer_size_factor" type = "double" >1.0</variable>
</variable_group>

<variable_group id = "parameters" >
<variable id = "using_server" type = "bool" >false</variable>
<variable id = "using_oasis" type = "bool">false</variable>
<variable id = "info_level" type = "int" >50</variable>
<variable id = "print_file" type="bool">true</variable>
<variable id = "oasis_codes_id" type="string" >lfric2lfric lfric2lfric</variable>
</variable_group>
</variable_definition>
</context>

</simulation>
1 change: 0 additions & 1 deletion rose-stem/app/lfric2lfric/opt/rose-app-clim_gal9.conf
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,4 @@ stretching_height=17507.0
stretching_method='smooth'

[namelist:files]
checkpoint_stem_name='restart_lfric2lfric_clim_gal9_C12_MG'
start_dump_filename='lfric2lfric_clim_gal9_C12_MG'
10 changes: 10 additions & 0 deletions rose-stem/app/lfric2lfric/opt/rose-app-dst_melbourne.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
[namelist:lfric2lfric]
destination_mesh_name='melbourne'
destination_meshfile_prefix='${MESH_DIR}/melbourne/mesh_melbourne'
destination_topology='non_periodic'
dst_ancil_directory='$BIG_DATA_DIR/ancils/proto-ral/Ticket255/Melbourne'
dst_orography_mean_ancil_path=''
target_domain='LAM'

[namelist:partitioning(destination)]
partitioner='planar'
10 changes: 10 additions & 0 deletions rose-stem/app/lfric2lfric/opt/rose-app-dst_ukv.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
[namelist:lfric2lfric]
destination_mesh_name='ukv'
destination_meshfile_prefix='${MESH_DIR}/ukv/mesh_ukv'
destination_topology='non_periodic'
dst_ancil_directory=''
dst_orography_mean_ancil_path=''
target_domain='LAM'

[namelist:partitioning(destination)]
partitioner='planar'
1 change: 0 additions & 1 deletion rose-stem/app/lfric2lfric/opt/rose-app-ral3.conf
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ stretching_height=16641.984
stretching_method='smooth'

[namelist:files]
checkpoint_stem_name='restart_lfric2lfric_ral3_seuk'
!!lbc_directory='$BIG_DATA_DIR/lbcs/proto-ral/Ticket3510/seuk'
!!lbc_filename='checkpoint_um2lfric_000001_fixedtime_date'
start_dump_filename='lfric2lfric_ral3_seuk'
Expand Down
30 changes: 30 additions & 0 deletions rose-stem/app/lfric2lfric/opt/rose-app-src_C224_L70.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
[file:iodef.xml]
source=$ROSE_SUITE_DIR/app/lfric2lfric/file/iodef_orography.xml

[file:source_data_C224.nc]
mode=symlink
source="/data/users/christine.johnson/lfric_data/lfric2lfric/startdump_C224.nc"

[namelist:extrusion]
domain_height=80000.0
method='um_L70_50t_20s_80km'
number_of_layers=70
stretching_height=17507.0
stretching_method='smooth'

[namelist:files]
start_dump_filename='source_data_C224'

[namelist:lfric2lfric]
source_mesh_name='dynamics'
source_meshfile_prefix='${MESH_DIR}/C224_MG/mesh_C224_MG'
src_ancil_directory='$BIG_DATA_DIR/ancils/basic-gal/yak/C224'
src_orography_mean_ancil_path='orography/gmted_ramp2/qrparm.orog.ugrid'

[namelist:orography]
orog_init_option='none'

[namelist:partitioning(source)]
panel_decomposition='custom'
panel_xproc=6
panel_yproc=6
30 changes: 30 additions & 0 deletions rose-stem/app/lfric2lfric/opt/rose-app-src_C896_L70.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
[file:iodef.xml]
source=$ROSE_SUITE_DIR/app/lfric2lfric/file/iodef_diag_file.xml

[file:source_data_C896.nc]
mode=symlink
source="/data/users/christine.johnson/lfric_data/lfric2lfric/rho_theta_C896_t1.nc"

[namelist:extrusion]
domain_height=80000.0
method='um_L70_50t_20s_80km'
number_of_layers=70
stretching_height=17507.0
stretching_method='smooth'

[namelist:files]
start_dump_filename='source_data_C896'

[namelist:lfric2lfric]
source_mesh_name='dynamics'
source_meshfile_prefix='${MESH_DIR}/C896_MG/mesh_C896_MG'
src_ancil_directory=''
src_orography_mean_ancil_path=''

[namelist:orography]
orog_init_option='none'

[namelist:partitioning(source)]
panel_decomposition='custom'
panel_xproc=16
panel_yproc=8
23 changes: 23 additions & 0 deletions rose-stem/app/mesh/opt/rose-app-melbourne.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
[command]
default=mkdir -p $MESH_DIR ; mpiexec -n 1 $BIN_DIR/planar_mesh_generator mesh_generation.nml

[env]
mesh_generator=Planar

[!!namelist:cubedsphere_mesh]

[namelist:mesh]
mesh_names='melbourne'
rotate_mesh=.true.
topology='non_periodic'

[namelist:planar_mesh]
domain_centre=144.578,-37.472
domain_size=3.456,3.456
edge_cells_x=256
edge_cells_y=256
periodic_x=.false.
periodic_y=.false.

[namelist:rotation]
target_north_pole=180.0,90.0
32 changes: 32 additions & 0 deletions rose-stem/app/mesh/opt/rose-app-ukv.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
[command]
default=mkdir -p $MESH_DIR ; mpiexec -n 1 $BIN_DIR/planar_mesh_generator mesh_generation.nml

[env]
mesh_generator=Planar

[!!namelist:cubedsphere_mesh]

[namelist:mesh]
mesh_names='ukv'
rotate_mesh=.true.
topology='non_periodic'

[namelist:planar_mesh]
domain_centre=-0.872500,1.6968
domain_size=1,1
edge_cells_x=960
edge_cells_y=1024
periodic_x=.false.
periodic_y=.false.
stretch_function='inflation'

[namelist:rotation]
target_north_pole=177.5,37.5

[namelist:stretch_transform]
cell_size_inner=0.0135,0.0135
cell_size_outer=0.036,0.036
n_cells_outer=140,85
n_cells_stretch=21,21
stretching_on='p_points'
transform_mesh='ukv'
45 changes: 45 additions & 0 deletions rose-stem/site/common/lfric2lfric/tasks_lfric2lfric.cylc
Original file line number Diff line number Diff line change
Expand Up @@ -216,6 +216,51 @@
"xios_min_buffer_size": 2097152,
}) %}

{% elif task_ns.conf_name == "oasis_C224_to_Melbourne" %}

{% do task_dict.update({
"opt_confs": ["oasis","src_C224_L70","dst_melbourne"],
"resolution": ["melbourne","C224_MG"],
"dst_mesh": "melbourne",
"dst_name": "melbourne",
"dst_type": "regional",
"src_mesh": "C224_MG",
"src_name": "dynamics",
"src_type": "global",
"mpi_parts": 216,
"kgo_checks": [],
}) %}

{% elif task_ns.conf_name == "oasis_C896_to_Melbourne" %}

{% do task_dict.update({
"opt_confs": ["oasis","src_C896_L70","dst_melbourne"],
"resolution": ["melbourne","C896_MG"],
"dst_mesh": "melbourne",
"dst_name": "melbourne",
"dst_type": "regional",
"src_mesh": "C896_MG",
"src_name": "dynamics",
"src_type": "global",
"mpi_parts": 768,
"memory": [250, "GB"],
"kgo_checks": [],
}) %}

{% elif task_ns.conf_name == "oasis_C896_to_ukv" %}

{% do task_dict.update({
"opt_confs": ["oasis","src_C896_L70","dst_ukv"],
"resolution": ["ukv","C896_MG"],
"dst_mesh": "ukv",
"dst_name": "ukv",
"dst_type": "regional",
"src_mesh": "C896_MG",
"src_name": "dynamics",
"src_type": "global",
"mpi_parts": 768,
"kgo_checks": [],
}) %}
{% elif task_ns.conf_name == "canned" %}

{% do task_dict.update({
Expand Down
5 changes: 5 additions & 0 deletions rose-stem/site/meto/groups/groups_lfric2lfric.cylc
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,11 @@
"lfric2lfric_oasis_clim_gal9_C12-ral_seuk_C16_lam_4cpu_ex1a_cce_fast-debug-64bit",
"lfric2lfric_oasis_clim_gal9_C12-ral_seuk_C16_lam-lbc_4cpu_ex1a_cce_fast-debug-64bit",
],
"lfric2lfric_ex1a_weekly": [
"lfric2lfric_oasis_C224_to_Melbourne_ex1a_cce_fast-debug-64bit",
"lfric2lfric_oasis_C896_to_Melbourne_ex1a_cce_fast-debug-64bit",
"lfric2lfric_oasis_C896_to_ukv_ex1a_cce_fast-debug-64bit",
],
"lfric2lfric_ex1a": [
"lfric2lfric_ex1a_developer",
"lfric2lfric_ex1a_extra",
Expand Down
2 changes: 1 addition & 1 deletion rose-stem/templates/default_directives.cylc
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
"tech-tests_memory": [6, "GB"],
"tech-tests_wallclock": 20,
"tech-tests_cpus": 1,
"gen_weights_tasks": 8,
"gen_weights_tasks": 12,
"gen_weights_memory": [16, "GB"],
} %}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,8 @@
'test -f *-checksums.txt && cp *-checksums.txt $TASK_OUTPUT_DIR/checksum.txt',
'touch $CYLC_TASK_WORK_DIR/tmp.m',
'mv $CYLC_TASK_WORK_DIR/*.m $TASK_OUTPUT_DIR/results',
'test -f $CYLC_TASK_WORK_DIR/lfric_checkpoint_write.nc && '~
'mv $CYLC_TASK_WORK_DIR/lfric_checkpoint_write.nc $TASK_OUTPUT_DIR/results || true',
'test -f $CYLC_TASK_WORK_DIR/timer.txt && '~
'cp $CYLC_TASK_WORK_DIR/timer.txt $TASK_OUTPUT_DIR || true'
] %}
Expand Down
Loading