Name a manifest that disagrees with the installed fwl-io - #32
Conversation
A model ships its manifest inside its own package, so the manifest can be newer than the fwl-io reading it. Until now an unknown field was silently ignored, which is the worst of the options: the manifest asked for something and got no error and no effect. A dataset field written one level too high was dropped the same way, so a `required_by` on the grouping table left the dataset claiming no model needs it and `fwl-io fetch <model>` quietly fetched nothing. A dataset table now declares only the fields fwl-io knows and a grouping level declares none, with anything else raising `ManifestSchemaError`. The message names the action that fits the case: move a dataset field that sits too high, delete a field this fwl-io no longer takes, and for a name it does not know at all, check the spelling or upgrade, since either can be the cause. The error subclasses ValueError, so callers that already treat a bad manifest as a ValueError are unaffected. The message identifies the reading code by the manifest schema number it implements, carried in the source, with the packaging version alongside it and labelled as such. An editable checkout keeps the version recorded at install time, so on the setup this ecosystem actually uses the packaging version can name a release containing none of the code that is running. The manifests page lists what each schema number means and the release it arrived in. Two things stay outside the check, and the documentation says so. A scalar at the manifest root is reserved for a manifest's own settings, which is what lets a manifest carry them beside its dataset tables. And a table is recognised as a dataset by its `zenodo` key, so a misspelt `zenodo` is reported as a table with no pin rather than as an unknown field.
A dataset field one level too high was refused on a grouping table but still dropped silently at the manifest root, which for a top-level dataset table is the same level above it. Both are refused now, with a message naming the move, while a root scalar that names no dataset field stays a manifest's own setting and still loads. That message also names the schema the reading code implements, as the other two already did, so every branch can be placed against the schema table the manifests page carries. The table lists the release in the form an install reports it. `ManifestSchemaError` joins the API reference, since it is exported, and its docstring now covers the third case it reports.
|
Pushed a correction round. The misplaced-field message did not name the schema, which made two sentences on the manifests page false; it does now, and so does the dropped-field message, with a test on each. A dataset field at the manifest root was still dropped silently, which for a top-level dataset table is the same level above it that the grouping check covers, so it is refused too; a root scalar naming no dataset field is still a manifest setting and still loads. One correction to the description above: the mutation I listed as "hardcoding it instead of reading it" is degenerate in an environment whose installed version happens to equal the hardcoded string. A hardcode with a different value is caught; that one is not evidence of anything. |
A model ships its manifest inside its own package, so a manifest can be newer than the fwl-io reading it. Until now an unknown field in a dataset table was silently ignored, which is the worst of the options: the manifest asked for something and got no error and no effect. A dataset field written one level too high was dropped the same way, so
required_byon the grouping table left the dataset claiming no model needs it andfwl-io fetch <model>quietly fetched nothing.ManifestSchemaError.ManifestSchemaErrorsubclassesValueError, so every existing caller keeps working unchanged:_discover,fetch_for,sync, the CLI, and the consumer-side handlers in MORS and PROTEUS.docs/Explanations/manifests.mdgains a schema table saying what each number means and the release it arrived in, and states the two things outside the check.Two exclusions are deliberate and documented. A scalar at the manifest root stays ignored, which is what lets a manifest carry its own settings beside its dataset tables. And a table is recognised as a dataset by its
zenodokey, so a misspeltzenodois reported as a table with no pin rather than as an unknown field.Compatibility. Rejecting unknown fields is stricter than before. Every manifest that exists today loads unchanged: the one fwl-io ships, MORS's, and PROTEUS's, all verified by loading them. A third-party manifest carrying an extra field would now fail, and because
discover_manifestsskips a provider whose manifest fails to load, it would lose that provider's other datasets too; the docs say so, andfwl-io listshows the error.Testing. Full suite 251 passed, 1 skipped.
ruff checkandruff format --checkclean. Ten mutations were run against the new tests and each is caught by exactly one: widening the accepted field set (including re-admittingsubdir), reporting only the first unknown field, truncating the known-field list, renumbering the schema silently, dropping the distribution version, hardcoding it instead of reading it, sending a misplaced known field down the spelling-or-upgrade path, dropping both actions from the unknown-field message, reverting thesubdirrejection to a plainValueError, and stripping the grouping-level error of the field and table names.