feat: bump polars 0.53 -> 0.55.2 - #41
Open
rob-p wants to merge 2 commits into
Open
Conversation
A clean checkout currently fails to build anndata-hdf5:
error: failed to run custom build command for `hdf5-metno-sys v0.11.3`
thread 'main' panicked at hdf5-metno-sys-0.11.3/build.rs:222:21:
Invalid H5_VERSION: "2.2.0"
Nothing here changed. hdf5-metno-src 0.10.3, published 2026-08-04, bumped the
bundled HDF5 to 2.2.0. hdf5-metno-sys 0.11.x depends on it as ^0.10 and so picks
it up automatically, but 0.11.x's build script accepts only HDF5 minor versions
{0,1,8,10,12,14} and rejects 2.2.0. The parser fix shipped in hdf5-metno-sys
0.12.2 only (metno/hdf5-rust#202).
Moving to 0.12.x is not currently possible from here: this crate requires both
hdf5-metno-sys ^0.11 and hdf5-metno ^0.12.3, and any hdf5-metno >= 0.12.6 needs
hdf5-metno-sys ^0.12. Since that crate declares `links = "hdf5"`, cargo refuses
to have both in the graph rather than choosing one.
Declaring hdf5-metno-src directly costs nothing: this crate already enables
hdf5-metno-sys's `static` feature unconditionally, so those sources are built in
every configuration regardless. The pin only chooses which of them.
Fixes scverse#38
polars is pre-1.0, so each minor release is a distinct set of types. anndata takes and returns polars DataFrames across its public API but does not re-export polars, so every downstream crate has to match this pin exactly -- which means the whole downstream ecosystem sits on whatever this crate chooses. The Python bindings drag the rest of the chain along: polars 0.53 -> 0.55.2 the actual goal pyo3-polars 0.26 -> 0.28 first release supporting polars 0.55 pyo3 0.27 -> 0.29 required by pyo3-polars 0.28 numpy 0.27 -> 0.29 tracks pyo3 in lockstep pyo3-log 0.13 -> 0.13.4 earlier releases cap at pyo3 < 0.29 The requirement has to be 0.55.2 rather than 0.55: polars 0.55.0 declares arrow = ^0.55.0 and polars-parquet = ^0.55.0, but polars-parquet 0.55.2 needs polars-arrow ^0.55.2, so resolving to 0.55.0 fails outright. The only source change is that polars 0.55 removed `impl IntoIterator for &ChunkedArray<T>`, so `.into_iter()` on a borrowed ChunkedArray becomes `.iter()`. Same items, same semantics; 14 call sites. pyo3 0.29 itself needed no changes. Closes scverse#39
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #39.
Why
polarsis pre-1.0, so each minor release is a distinct, incompatible set of types.anndatatakes and returnspolars::prelude::DataFrameacross its public API but does not re-exportpolars, so every downstream crate must declare apolarsdependency matching this one exactly. That means this pin decides the polars version for the whole downstream ecosystem, and it is currently two minor versions behind.What changed
polarspyo3-polarspyo3pyo3-polars0.28numpypyo3in locksteppyo3-logpyo3 < 0.29Two things worth knowing:
0.55.2, not0.55. polars 0.55.0 declaresarrow = "^0.55.0"andpolars-parquet = "^0.55.0", butpolars-parquet0.55.2 requirespolars-arrow ^0.55.2, so a resolution that picks 0.55.0 fails withfailed to select a version for polars-arrow.pyo30.29 required no source changes. The bump is purely a manifest change on that side.The only code change is that polars 0.55 removed
impl IntoIterator for &ChunkedArray<T>, so.into_iter()on a borrowedChunkedArraybecomes.iter(). Same items (Option<T>), same semantics. 14 call sites:anndata/src/data/array/dataframe.rs(11)anndata/src/anndata/dataset.rs(1)pyanndata/src/anndata/backed.rs(1)pyanndata/src/anndata/dataset.rs(1)Verified
rustc 1.97.1, x86_64-unknown-linux-gnu, Python 3.13.7, cmake 3.29.2.
I have not exercised the Python test suite — I could only get as far as compiling
pyanndataandanndata_rs— so a look from someone who can run the bindings end-to-end would be worthwhile before merging.Separately worth considering
Not included here, but it would retire this whole class of problem for downstreams:
Consumers could then use
anndata::polars::prelude::DataFrameand be structurally unable to resolve a mismatched version, instead of hand-shadowing this crate's choices and re-checking them each release. Both types already cross the public API boundary (DataFrameviaset_obs/set_var,CsrMatrixviaDynCsrMatrix). Happy to add it here or in a follow-up if you want it.