Blosc2 filter fails writes (and can crash at file close) under HDF5 2.2.0

[jrideout]

HDF5 2.2.0 (released 2026-07-29) added a consistency check to the filter pipeline: a filter callback that returns a data size larger than its reported buffer size now fails hard with buffer size is too small after filter callback (H5Z.c:1607 on write, :1518 on read; absent in <= 2.1.1).

The Blosc2 filter vendored from HDF5-Blosc2 trips this check. Its compress path reports *buf_size as the precomputed uncompressed-chunk-size guess from cd_values[3] (blosc2_filter.c:296 → :592) while returning the actual size of the Blosc2 cframe it produced. A cframe carries fixed format overhead, so whenever a chunk compresses poorly — incompressible data, or small chunks where the overhead dominates — the returned size exceeds the reported buffer size. Under HDF5 <= 2.1.1 this bookkeeping mismatch was harmless (the buffer really is the larger size); under 2.2.0 the chunk flush fails.

Through h5py the symptom is nasty: chunk flushes happen at file close, so the error surfaces during ObjectID.__dealloc__ ("Can't decrement id ref count (buffer size is too small after filter callback)"), the affected chunks are silently lost, and the interpreter can crash (SIGSEGV) during teardown.

Reproducer (hdf5plugin 7.0.0 wheel, h5py built against libhdf5 2.2.0, macOS arm64 — but nothing platform-specific):

import numpy as np, h5py, hdf5plugin

rng = np.random.default_rng(0)
data = rng.integers(0, 256, (100, 100), dtype=np.uint8)  # incompressible

with h5py.File("repro.h5", "w") as f:
    f.create_dataset("x", data=data, chunks=(100, 100),
                     **hdf5plugin.Blosc2(cname="zstd", clevel=1))
# -> RuntimeError at close ("buffer size is too small after filter callback"),
#    then SIGSEGV at interpreter exit; chunks are never written.

Compressible data (e.g. zeros, or random floats in [0, 1), which byte-shuffle well) writes fine, which is why routine testing does not catch it. The other ten compression filters shipped by hdf5plugin (blosc, bitshuffle, lz4, zstd, zfp, bzip2, SZ, SZ3, sperr, fcidecomp) report buffer sizes consistently and are unaffected — I checked each compress path against the new rule.

The fix is one line — report the real size of the returned frame — proposed upstream at Blosc/HDF5-Blosc2#5; PyTables applied the same fix to its vendored copy in PyTables/PyTables#1347.

Exposure today is limited because no major distribution ships HDF5 2.2.0 yet (conda-forge is on 2.1.0), but h5py has a pending PR to bundle 2.2.0 in its wheels (h5py/h5py#2947), at which point any pip user combining h5py + hdf5plugin.Blosc2 would hit this.