Define a schema for workflow config files

[victorlin]

Note

This is different from standardizing the schema itself across pathogen repos.

This has been mentioned at least a few times and it would be good to start exploring how to do this.

In nextstrain/pathogen-repo-guide#74:

We’ve played with JSON schemas (for the config) and auto-generating HTML docs from this. That effort wasn’t successful enough to merge but I think this will be where we eventually end up.

In nextstrain/WNV#105 (comment):

Use a JSON Schema for the config to figure out what to resolve automatically and then do so. This lets the code be shared while each pathogen gets path resolution "for free" by adding a config schema. We want schemas for our configs anyway. Why not use this as another carrot and reduce copying at the same time?

In nextstrain/rsv#106 (comment):

Maybe this is another +1 for defining a config schema, which could be a basis for documentation that tells users to use custom_subsample instead of subsample.

Options to explore

• JSON Schema
• Snakemake's built-in validation
• Augur's built-in validation
• Surface schema to users
  • Generate docs page?
  • Use $schema keyword for better editing experience in our default configs? User configs?

Progress

• Configuration schema and validation measles#88
• Skip schema validation when using custom rules measles#145
• Validate schema locally measles#141
• Configuration schema and validation rsv#139
• Move dump_and_validate from measles and rsv to nextstrain/shared
• Add schema to more repos

[tsibley]

The starting point here is clear. We either write a JSON Schema or, likely better, write a schema in CUE and generate a JSON Schema from that as needed for other tooling, like Snakemake.

The procrastination on config schemas hasn't been so much not knowing how or where to start, but in simply the activation energy to start it and see it thru. I think once we have complete config schemas they'll be regularly useful enough that we'll wonder why we didn't write them sooner.

Note that ncov has had a (very partial, so not super useful) config schema for a long time.

[victorlin]

Conceptually, I think there are two schemas:

1. Workflow schema
   • This is what Snakemake expects in the config variable – the outcome of merging the default config with any additional --configfiles.
2. User schema
   • This is what can be given to --configfile – since defaults are exhaustive, this should be (1) with all properties marked as optional.

We could define a single schema file for (1) and use that for validation directly on the config variable, but any user-facing docs should first transform it to mark all properties as optional.

[victorlin]

An example of validation which could be improved with a schema (but maybe not entirely replaceable):

def validate_config():
    """
    Validate the config.

    This could be improved with a schema definition file, but for now it serves
    to provide useful error messages for common user errors and effects of
    breaking changes.
    """
    # Validate 'builds'
    if invalid_builds := set(config["builds"]) - set(config["build_params"]):
        print(dedent(f"""\
            ERROR: The following names in 'builds' are not defined in 'build_params':

                {indented_list(invalid_builds, "            ")}

            Available builds are:

                {indented_list(config['build_params'], "            ")}
            """))
        exit(1)

    # Check for deprecated 'subsampling' keys
    for build in config["builds"]:
        if "subsampling" in config["build_params"][build]:
            print(dedent(f"""\
                ERROR: The 'subsampling' configuration key is no longer supported.

                Please rename it to 'subsample' and update the format to match
                the augur subsample --config structure:

                    <https://docs.nextstrain.org/projects/augur/en/stable/usage/cli/subsample.html#configuration>"""))
            exit(1)

[victorlin]

If we choose to modify the config structure as discussed in #23, we may want schemas for both the raw, unmodified config as well as the processed config.

[victorlin]

I created this issue under nextstrain/pathogen-repo-guide, but schemas are repo-specific. Transferring to nextstrain/public.