Describe interoperability with a few generic goals

[benbovy]

I've been thinking a bit more about this, trying to gather all the relevant suggestions (currently on the website in the "Goals" section and also in various issues #169, #183, etc.) into fewer, more abstract concepts and goals. Here is an attempt (this could certainly be improved in many ways):

1. "Human" interoperability: help users making sense of the model's data and metadata so that they can easily relate it to other models or observations. Reduce cognitive load and the mental overhead of context switching.

2. Machine (data/metadata) interoperability: enable seamless data/metadata transfer between a model software and other (model) software.

3. Machine (program) interoperability: allow model execution being controlled by another software (either coarse-grained or fine-grained control).

How would each of those goals relate / translate into standards or good practices ? Some examples:

1. Documentation, compliance with standardized data models or conventions (e.g., CDM, CF-conventions), standard names...

2. Standard (or at least common and portable) storage formats that can be easily read/written by machines, e.g., NetCDF4, HDF5, CSV, Parquet, JSON, etc.

3. Standardized APIs and protocols, compliance / integration with modelling frameworks and/or scientific computing ecosystems...

Note that a particular standard may be useful for more than one goal. Some examples:

• Using a standard data model or convention is often helpful for goal 2 too (a generic storage format is not enough for a machine or software to fully "make sense" of the transferred data)
• Using common storage formats like YAML or TOML for model configuration files may be helpful for goal 1 too, as those formats could be easily read by both humans and machines
• Using well-designed, standardized interfaces (UI or API) across model implementations might greatly reduce cognitive load (goal 1)

[platipodium]

I really like this generalization. I would combine the Machine interoperability items into one

Machine (data/metadata) interoperability: enable seamless data/metadata transfer between a model software and other (model) software.
Machine (program) interoperability: allow model execution being controlled by another software (either coarse-grained or fine-grained control).

Machine interoperability: provide sufficient metadata to ensure seamless data transfer between models and to communicate the control flow between models.

Then, we might look at a third point, which is really portability, i.e. the deployment of a model on different machines.

[benbovy]

I like the idea of merging the "machine interoperability" items! I'm not sure about "provide sufficient metadata...", though, as IMO machine interoperability depends on more than that (at least for communicating control flows), e.g., on how models are designed and implemented.

Then, we might look at a third point, which is really portability, i.e. the deployment of a model on different machines.

I was also thinking about it, but it still not clear to me where this should go. For example, one could imagine two models deployed on their specific platforms or machines, thus not portable, but still able to communicate to each other, e.g., over the network via REST APIs. What matters the most in this case is for the two models to have a standardized interface.

That said, it seems rather obvious to me that portable applications and portable I/O storage formats greatly facilitate interoperability between models, from both the human and machine points of view.

I'm just wondering whether portability should be a goal here or rather one requirement that helps to achieve the goals.

[platipodium]

machine interoperability depends on more than that (at least for communicating control flows), e.g., on how models are designed and implemented.

Yes, it depends on communicating metadata on data and control facilities, and on accepting data and control information.