Skip to content

Conversation

@olivhoenen
Copy link
Collaborator

olivhoenen and others added 30 commits March 20, 2025 15:33
Select `imas_core` when the URI starts with `imas:`, otherwise use the
netCDF backend.

This change allows using NCZarr [1] as storage engine (when it is
enabled in the netCDF4 python module) by using `file://`, `s3://` or
`https://` style URIs.

[1] https://docs.unidata.ucar.edu/nug/current/nczarr_head.html
Pytest runs successfully with numpy 2.x, so we should be able to remove this restriction.
We request `ids_properties/homogeneous_time` and
`ids_properties/version_put/data_dictionary` in two separate calls
to the backend before actually getting all data. This is fine for local
backends, but UDA would fetch the data three times:

1. When determining the DD version and if the IDS exists
2. When determining whether the IDS uses homogeneous time
3. When actually reading the data

This commit adds a `datapath="ids_properties"` to the first two cases.
This results in UDA only fetching the IDS properties in points 1 and 2.
The full IDS is now requested once by UDA, in point 3.
…ert_ids call

This prevents polluting the output of the CLI `imas convert` and explict `imas.convert_ids` calls with lots of repeated warnings.
Co-authored-by: Maarten Sebregts <110895564+maarten-ic@users.noreply.github.com>
The Data Dictionary definitions are now provided by the `imas-data-dictionaries` package.
Co-authored-by: Maarten Sebregts <110895564+maarten-ic@users.noreply.github.com>
To migrate deprecated fields. See #55 for more details.
- Catch when an IDS is new in 3.42.0 before attempting to migrate obsolete nodes
- Fix issue with migrating obsolete structures and AoS (
maarten-ic and others added 16 commits October 13, 2025 13:55
…ions on put()

And fix an example in the multi-DD documentation.
Implicit conversion between major versions of the DD is almost always giving incorrect results, so it's better to disallow this instead of emitting warnings (which people usually don't read).

This commit changes the warnings that were previously emitted into runtime errors. The errors refer to the (updated) documentation, which provides an example that does work correctly.
- Errorbars were not cleared by `unset_coordinate()`, this is fixed now
- Use imas.util.tree_iter in unset_coordinate, which is more clear than visit_children and performs slightly better
- Simplify some checks in `maybe_set_random_value()`
Copy link
Collaborator

@maarten-ic maarten-ic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some small comments. Could you please fix CI as well before tagging the release?

@olivhoenen
Copy link
Collaborator Author

Some small comments. Could you please fix CI as well before tagging the release?

Indeed, tests were failing because the mdsplus backend is not available in the current imas_core wheel on pypi. I've added an ugly to check that.

@olivhoenen olivhoenen merged commit a3bba0b into main Dec 12, 2025
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants