Add Fab build script

[hiker]

Add a first Fab-based build script.

[hiker]

The file https://github.com/MetOffice/socrates/blob/main/src/odepack/opkda1.f is causing parsing issues with fparser, fparser takes over 30 minutes to parse this file. Issue raised at stfc/fparser#499

We might have to implement a custom work around for this file, depending on the above ticket :(

[hiker]

While I have thought I had fixed stfc/fparser#499, apparently this fix only works for F2003, while Fab is using F2008 (which still has the issue). Continuing to investigate this. Alternative solution: allow an app to switch the Fortran standard (which Fab already can, we just need to expose this in the base class) to use F2003.

[hiker]

Additional issue: Socrates uses include statements to include one Fortran file into another, e.g.:

less seaalbedo_driver.f
...
      include 'seaalbedo.f'

This causes Fab to report duplicated symbols (rightly). We need to either exclude the included files from the build process (though they still need to be copied into the build tree so that the include works. Note that this is NOT a preprocessor include, but a Fortran include :( ). Alternative: remove the includes, and fix up the source code?

While there are a lot of include (over 80), most are specific .finc files, which will not be parsed. seaalbedo above seems to be the main issue, though Fab also reported many nc duplications, which need to be investigated.

[hiker]

While I have thought I had fixed stfc/fparser#499, apparently this fix only works for F2003, while Fab is using F2008 (which still has the issue). Continuing to investigate this. Alternative solution: allow an app to switch the Fortran standard (which Fab already can, we just need to expose this in the base class) to use F2003.

OK, that was actually an easy fix: Fortran 2008 has a new/different rule for Label_Do_Stmt, including this fixed the performance issue for F2008 as well.

[hiker]

I've submitted MetOffice/fab#558 to fix the handling of 'containedsubroutines. Excluding theinclude`d file as well (see above) allowed compilation to get started.

While adjusting (downgrading really) the compilation option, I did catch at least one issue that feels like a real problem to be in src/aux/run_distort.f:

       IF ( (i_angular_integration == IP_two_stream).OR.
      &     (i_angular_integration == IP_ir_gauss).OR.
      &     ( (i_angular_integration == IP_spherical_harmonic).AND.
     &       (i_sph_flux_mode == IP_sph_mode_flux) ) ) THEN
 !
         IF (l_non_net) THEN
           DO i=0, n_layer

There doesn't appear to be a i_sph_flux_mode variable anywhere in Socrates. Gfortran helpfully(?) suggests:

Error: Symbol ‘i_sph_flux_mode’ at (1) has no IMPLICIT type; did you mean ‘i_sph_mode’?

What should I do with that?

[mo-jmanners]

Hi Joerg,

Are you intending to use Fab to compile Socrates for building it in to LFRic / UM? The majority of the Socrates codebase is used for research purposes and not intended for operational use. The bits that have been causing you trouble aren't built into LFRic so you probably don't need to be compiling these at all. On the specific issues:

1. opkda1.f: this is part of the publicly available ODEPACK code that is imported into Socrates for use with the chemistry code in src/flexchem/. This is not built into trunk LFRic but is used with the UM on a branch at Exeter University. It is intended for incorporation into LFRic at some point in the future. We use extra compiler options for gfortran to compile it: -w -std=legacy -fallow-argument-mismatch
2. seaalbedo.f is used as a bit of a hack in the offline Socrates driver l_run_cdf. It will never be built into LFRic/UM. I don't think seaalbedo.f is used anywhere else apart from seaalbedo_driver.f so we could just paste it in there directly and get rid of the include.
3. run_disort.f has not been used / compiled for a number of years (it is not included in the offline makefiles). It has been kept to act as an implementation guide for users that may want to run DISORT in the future. We could move this to a file without the .f extension (maybe README_DISORT) if that helps.

James

[hiker]

Hi James,

thanks a lot for your help and information provided, really appreciated!!

I am getting the feeling that there is a bit of a communication breakdown with Yash? I'll send a separate email to you and Yash, and just answer technical issues here.

I am using this ticket to keep track of the issue I experience while developing the script, and e.g. to set priorities for Fab PRs based on where these changes are required.

Are you intending to use Fab to compile Socrates for building it in to LFRic / UM? The majority of the Socrates codebase is used for research purposes and not intended for operational use. The bits that have been causing you trouble aren't built into LFRic so you probably don't need to be compiling these at all. On the specific issues:

The UM/LFRic components are already supported by the corresponding UM/LFRic Fab scripts (only UM is merged so far, LFRic is still under review), so this is for building the binaries that are part of Socrates.

1. opkda1.f: this is part of the publicly available ODEPACK code that is imported into Socrates for use with the chemistry code in src/flexchem/. This is not built into trunk LFRic but is used with the UM on a branch at Exeter University. It is intended for incorporation into LFRic at some point in the future. We use extra compiler options for gfortran to compile it: -w -std=legacy -fallow-argument-mismatch

Yes, the problem around this file have been solved by now. It triggered an interesting bug in fparser (PR still to be reviewed, but the issue is fixed), so we will need a new release of fparser, which we can probably trigger on short notice.

3. seaalbedo.f is used as a bit of a hack in the offline Socrates driver l_run_cdf. It will never be built into LFRic/UM. I don't think seaalbedo.f is used anywhere else apart from seaalbedo_driver.f so we could just paste it in there directly and get rid of the include.

Indeed, that file is not used at all, and I am excluding it. I might explain a bit of Fab: I just pointed Fab at the sources and told it to find and build all binaries ;) As part of this is parses all Fortran files (since only after that it can know if a file is used or not). Including a full Fortran file was not something that was considered in Fab (it will report a duplicated symbol, since indeed the subroutine would be compiled twice). Excluding this file solved this issue easily.

5. run_disort.f has not been used / compiled for a number of years (it is not included in the offline makefiles). It has been kept to act as an implementation guide for users that may want to run DISORT in the future. We could move this to a file without the .f extension (maybe README_DISORT) if that helps.

Similarly, I have simply excluded this file as well.

ATM, I have around 24 binaries compiled with the Fab script. I need to fix external handling 'more' in Fab (MetOffice/fab#559 (comment)), then I expect the remaining binaries should also link.

[hiker]

I have all binaries built (I think :) ):

Cinc_field           cdf2samson           dentomix         fval           ice_scatter  l_run_cdf    nettohr           raw_input         scale_field          tidy_90
Cscale_field         cdl2cdf              dentomix_cdf     gen_horiz_cdl  ice_size     l_run_cdl    orb_elem          reduce_casim_ice  scatter_90           tune_weights
assign_subcols       cdl2samson           dentoq           gen_null       icedb2bin    lblcat       phgt              run_mono          scatter_average_90   vert_int
bdm2xsc              convert_elsey_shine  dewtoq           gen_surf_cdl   inc_field    make_filter  phoenix2sol       runes_driver      ser2uvxsc            wadd
build_output         corr_k               flexchem_driver  gen_view_cdl   interp       merge        prep_opt_profile  runes_nc          ser2xsc              xsc2xsc
calc_horizon_angles  cosp_driver          format_size      genln2cdf      interp_cdf   metrics      prep_spec         samson2cdf        source
cdf2cdl              dat2xsc              ft2qy            hmrsph         jpl2xsc      mid_point    qtxt2parsum       samson2cdl        sra_aerosol_profile

(ignore build_output, metrics, and source, which are Fab-specific directories).

I have applied a few changes to source code, which triggered issues elsewhere, and which are much easier to fix here:

• added some external declarations
• fixed some comments to start at column 0
• inlined seaalbedo.f into seaalbedo_driver.f (to remove the include statement that did this). This caused problems with the dependency analysis. Alternative, we could use #include and leave it to the preprocessor if this is preferable.

Note that this needs three additional PRs to be merged into Fab (for now I have a separate branch fab_for_socrates). I am waiting for additional feedback from Yaswant Pradhan (@yaswant) and James Manners (@mo-jmanners) via email to confirm that I should proceed with this.

[hiker]

All work so far is in the branch 29 add fab build script