Skip to content

feat: bump polars 0.53 -> 0.55.2 - #41

Open
rob-p wants to merge 2 commits into
scverse:mainfrom
rob-p:feat/polars-0.55
Open

feat: bump polars 0.53 -> 0.55.2#41
rob-p wants to merge 2 commits into
scverse:mainfrom
rob-p:feat/polars-0.55

Conversation

@rob-p

@rob-p rob-p commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

Closes #39.

Stacked on #40. The first commit here is that PR's one-line build fix, without which anndata-hdf5 does not compile at all and CI cannot go green. Merge #40 first and I will rebase, or take both together — the polars work is the second commit.

Why

polars is pre-1.0, so each minor release is a distinct, incompatible set of types. anndata takes and returns polars::prelude::DataFrame across its public API but does not re-export polars, so every downstream crate must declare a polars dependency matching this one exactly. That means this pin decides the polars version for the whole downstream ecosystem, and it is currently two minor versions behind.

What changed

crate from to why
polars 0.53 0.55.2 the actual goal
pyo3-polars 0.26 0.28 first release supporting polars 0.55
pyo3 0.27 0.29 required by pyo3-polars 0.28
numpy 0.27 0.29 tracks pyo3 in lockstep
pyo3-log 0.13 0.13.4 earlier releases cap at pyo3 < 0.29

Two things worth knowing:

  • The requirement has to be 0.55.2, not 0.55. polars 0.55.0 declares arrow = "^0.55.0" and polars-parquet = "^0.55.0", but polars-parquet 0.55.2 requires polars-arrow ^0.55.2, so a resolution that picks 0.55.0 fails with failed to select a version for polars-arrow.
  • pyo3 0.29 required no source changes. The bump is purely a manifest change on that side.

The only code change is that polars 0.55 removed impl IntoIterator for &ChunkedArray<T>, so .into_iter() on a borrowed ChunkedArray becomes .iter(). Same items (Option<T>), same semantics. 14 call sites:

  • anndata/src/data/array/dataframe.rs (11)
  • anndata/src/anndata/dataset.rs (1)
  • pyanndata/src/anndata/backed.rs (1)
  • pyanndata/src/anndata/dataset.rs (1)

Verified

cargo check --workspace                                  # clean
cargo test -p anndata -p anndata-hdf5 -p anndata-zarr    # 19 passed; 0 failed

rustc 1.97.1, x86_64-unknown-linux-gnu, Python 3.13.7, cmake 3.29.2.

I have not exercised the Python test suite — I could only get as far as compiling pyanndata and anndata_rs — so a look from someone who can run the bindings end-to-end would be worthwhile before merging.

Separately worth considering

Not included here, but it would retire this whole class of problem for downstreams:

pub use nalgebra_sparse;
pub use polars;

Consumers could then use anndata::polars::prelude::DataFrame and be structurally unable to resolve a mismatched version, instead of hand-shadowing this crate's choices and re-checking them each release. Both types already cross the public API boundary (DataFrame via set_obs/set_var, CsrMatrix via DynCsrMatrix). Happy to add it here or in a follow-up if you want it.

rob-p added 2 commits August 10, 2026 15:22
A clean checkout currently fails to build anndata-hdf5:

    error: failed to run custom build command for `hdf5-metno-sys v0.11.3`
      thread 'main' panicked at hdf5-metno-sys-0.11.3/build.rs:222:21:
      Invalid H5_VERSION: "2.2.0"

Nothing here changed. hdf5-metno-src 0.10.3, published 2026-08-04, bumped the
bundled HDF5 to 2.2.0. hdf5-metno-sys 0.11.x depends on it as ^0.10 and so picks
it up automatically, but 0.11.x's build script accepts only HDF5 minor versions
{0,1,8,10,12,14} and rejects 2.2.0. The parser fix shipped in hdf5-metno-sys
0.12.2 only (metno/hdf5-rust#202).

Moving to 0.12.x is not currently possible from here: this crate requires both
hdf5-metno-sys ^0.11 and hdf5-metno ^0.12.3, and any hdf5-metno >= 0.12.6 needs
hdf5-metno-sys ^0.12. Since that crate declares `links = "hdf5"`, cargo refuses
to have both in the graph rather than choosing one.

Declaring hdf5-metno-src directly costs nothing: this crate already enables
hdf5-metno-sys's `static` feature unconditionally, so those sources are built in
every configuration regardless. The pin only chooses which of them.

Fixes scverse#38
polars is pre-1.0, so each minor release is a distinct set of types. anndata
takes and returns polars DataFrames across its public API but does not re-export
polars, so every downstream crate has to match this pin exactly -- which means
the whole downstream ecosystem sits on whatever this crate chooses.

The Python bindings drag the rest of the chain along:

  polars       0.53 -> 0.55.2   the actual goal
  pyo3-polars  0.26 -> 0.28     first release supporting polars 0.55
  pyo3         0.27 -> 0.29     required by pyo3-polars 0.28
  numpy        0.27 -> 0.29     tracks pyo3 in lockstep
  pyo3-log     0.13 -> 0.13.4   earlier releases cap at pyo3 < 0.29

The requirement has to be 0.55.2 rather than 0.55: polars 0.55.0 declares
arrow = ^0.55.0 and polars-parquet = ^0.55.0, but polars-parquet 0.55.2 needs
polars-arrow ^0.55.2, so resolving to 0.55.0 fails outright.

The only source change is that polars 0.55 removed
`impl IntoIterator for &ChunkedArray<T>`, so `.into_iter()` on a borrowed
ChunkedArray becomes `.iter()`. Same items, same semantics; 14 call sites.
pyo3 0.29 itself needed no changes.

Closes scverse#39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bump polars 0.53 -> 0.55

1 participant