(Contact: leighton.regayre@metoffice.gov.uk)
This project is licensed under the terms of the 3-Clause BSD (BSD3) license (See LICENSE.txt)
Model-to-observation comparisons (monthly mean) Model-to-model comparisons (monthly mean)
All 12 months of model data will be available for specified model years If absent, user will need to create soft, symbolic links for each variable with naming convention that provides the toolkit with pseudo-month files
STASH diagnostic outputs from Met Office model required to make efficient use of the toolkit are summarised in 'Stash_requirements_csv.csv' Original Excel spreadsheet availble from contact - Excel spreadsheet allows for additions and sorting Contains use case and i_mode_setup dependencies - note the toolkit works on i_mode_setup values 2, 8, 10, 11, 12 (all options at Jun 2026)
For model-obs comparisons, recommend using nudged simulations (at least to horizontal winds above around 2km) for a single year GASSP dataset has high concentration of observational data in some years (e.g. 2008)
All observation data files have been processed from freely available sources and include metadata and filenames that direct users towards data providers. Data providers should be cited in any publications where results are significantly dependent on original data - ideally all data providers in metadata of each file should be cited, even if only in a publication Appendix.
Collobrators will have read access to the repository If, as a collaborator, you are interested in helping improve the evaluation toolkit, please take a fork of the main branch, make your own branches for distinct changes if you plan to edit/add files of that fork then make a pull request for changes useful to others. Any changes will be reviewed and included by repository admin.
Please retain data structure preferences (nested dictionaries for model data, pandas dataframes for observational data, etc.) and please ensure any new functionality accomodates both model-model and model-obs applications (see 'The namelist file')
The toolkit should be pulled from github with the name 'aerosol_evaluation_toolkit' Each python file that relies on relative imports contains a short section of code that searches for 'aerosol_evaluation_toolkit' within parent directories and when found adds the parent directory to sys.path - this allows for relative imports to start with aerosol_evaluation_toolkit
All folders contain __init__.py which signals to python that these directories and sub-directories should be treated as packages and sub-packages Thus, everything in the evaluation toolkit works within the aerosol_evaluation_toolkit directory
Python cannot import from outside the package. So, soft, symbolic links are used to point to external files owned and altered by users
will be referred to through documentation as "github_root_path" The aerosol evaluation toolkit accesses directories in the user's workspace relative to the github_root_path.
contains scripts and a README.md file describing how to extract STASH items and prepare for use within the toolkit
can be run from the command line where suite-ids and model years will be sourced from namelist file (see below) e.g. > python pre_check_stash.py and will summarise details of any missing stash items (due to suite setup or extraction errors) Additional arguments $1, $2 and $3 are names of full, summarized and exceptions to stash error files (default .txt files exist)
controls the aerosol evaluation toolkit. This script creates and submits a slurm batch file to the local queue
github_root_path/DRIVER_aerosol_evalution_toolkit_jasmin.sh for Jasmin github_root_path/DRIVER_aerosol_evalution_toolkit_azure_spice.sh for Azure Spice github_root_path/DRIVER_aerosol_evalution_toolkit_azure_spice_array.sh for expedient workflow on Azure Spice
Multiple aerosol evaluation options exist and this will grow over time:
- aerosol size distribution (compared to GASSP data and multiple marine locations)
- aerosol optical depth (380 nm and 1020 nm)
- aerosol near-surface number concentration (N3, N30, N50)
- species near-surface concentrations (including nitrate)
Choice of $1 options (command line argument) will indicate if evaluation runs from 0==sbatch queue (recommended) 1==command line (for testing)
The driver script must be run from within the github_root_path e.g. ./DRIVER_aerosol_evalution_toolkit_XXX.sh 0
User can edit:
- what the soft, symbolic link namelists/namelist_aerosol_evaluation_toolkit_symbolic_link.py points to this should be a copy of github_root_path/namelists/namelist_aerosol_evaluation_toolkit_EXAMPLE.py (see below) in the user's own workspace.
- 'work_dir', where a directory 'queue_logs' slurm output will be created
- name of sbatch file if appropriate
- header information in the sbatch setup, such as job-name
- which elements of the aerosol_evaluation_toolkit will be used (comment out unwanted)
Note: order in which evaluation_functions are processed is irrelevant, but aerosol_size_distribution and plot_N_x_vs_GASSP_map cannot run in parallel, as both read/create identical files (as necessary) so memory allocation clash would be possible
is used to specify key aspects of the evaluation, such as model suite_ids, years to be processed, etc. The location of the namelist file is specified in the driver script (DRIVER_aerosol_evalution_toolkit_jasmin.sh)
The user must copy github_root_path/namelists/namelist_aerosol_evaluation_toolkit_EXAMPLE.py to a location matching the path specified in the driver script (github_root_path/DRIVER_aerosol_evalution_toolkit_jasmin.sh)
The namelist file is used by all aerosol evaluation scripts to create a dictionary of properties for the aerosol evaluation The following should be edited:
- model_paths / suite_ids / run_name: append the first and consectuive (as needed) model data locations and descriptions
- github_root_path (where repository was pulled)
- root_plot_path (external location where figures will be saved within sub-directories)
- model_version (informative user-defined names for file names and figure titles - no spaces)
- model_years (list of lists of years for model evaluation - must match model basis - distinct for each suite-id)
- observation_years (list of years to scan for observational data)
- comparison_type (choice of: model-obs / model-model / model-model-obs)
- free_or_nudged (text used in figure titles due to its importance for interpretation of toolkit figures)
- l_average_obs_years (list of logicals that determine if observation data will be averaged across years - may vary b/w model suite-id and sometimes overwritten as needed (e.g. for aerosol size distribution, always plot against observations, though with less detail for model-model)
- l_average_model_years (list of logicals to determine if model output will be averaged across years) - again, may be overwritten as needed, but default used where appropriate
- i_mode_setup (list of i_mode_setup values used in each suite)
- seasonal_only (T/F to indicate if figures should be seasonal mean only, or all months and seasons)
- l_N50_augment (T/F to indicate choice to include N50 data processed from aerosol size distributions - non-GASSP source with reference in metadata)
The namelist pointed to by the soft, symbolic link namelists/namelist_aerosol_evaluation_toolkit_symbolic_link.py
and content of the DRIVER script github_root_path/DRIVER_aerosol_evalution_toolkit_jasmin.sh
can be immediately updated for parallel use of the aerosol evaluation toolkit.
So long as the sbatch job-name is updated, a new batch script will be created and submitted. Parallel implementation of identical namelist is not recommended, due to errors associated with saving files created by the toolkit However, any change in namelist dictionary items [suite_id / model_version / model_years] will allow both uses of toolkit to complete