Prepare raw2features v0.1.1

.github/workflows/ci.yml

@@ -6,25 +6,42 @@ on:
   pull_request:
 
 jobs:
+  core-smoke:
+    # A real lean install: no default dev group and therefore no zarr, image, or
+    # torch stack. This protects the documented `pip install raw2features` path.
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v5
+        with:
+          persist-credentials: false
+      - uses: astral-sh/setup-uv@v7
+        with:
+          python-version: "3.12"
+      - name: Install (core only)
+        run: uv sync --locked --no-default-groups
+      - name: Smoke import and CLI
+        run: |
+          uv run --no-sync python -c "import raw2features, sys; assert 'torch' not in sys.modules; assert 'zarr' not in sys.modules"
+          uv run --no-sync raw2features --help
+
   lint-and-test:
-    # Core-only: lint + run the suite with no heavy extras installed. Proves the
-    # package imports and the core paths work without torch/zarr (heavy-dep tests
-    # self-skip), so an end user who installs just the core isn't broken. Run across
-    # every supported Python (matches the requires-python / classifiers contract).
+    # Default developer environment: lint + the offline suite across every
+    # supported Python. The dev group intentionally includes the light Zarr/image
+    # test stack; the separate core-smoke job protects the dependency-lean install.
     runs-on: ubuntu-latest
     strategy:
       fail-fast: false
       matrix:
-        python-version: ["3.11", "3.12", "3.13"]
+        python-version: ["3.11", "3.12", "3.13", "3.14"]
     steps:
       - uses: actions/checkout@v5
         with:
           persist-credentials: false
       - uses: astral-sh/setup-uv@v7
         with:
           python-version: ${{ matrix.python-version }}
-      - name: Install (core only)
-        run: uv sync
+      - name: Install (developer test environment)
+        run: uv sync --locked
       - name: Lint
         run: uv run ruff check .
       - name: Test (core)
@@ -45,6 +62,34 @@ jobs:
         with:
           python-version: "3.12"
       - name: Install (full stack, CPU torch)
-        run: uv sync --extra zarr --extra image --extra torch --extra models --extra stain
+        run: uv sync --locked --extra zarr --extra image --extra torch --extra models --extra stain
       - name: Test (with torch)
         run: uv run pytest -m "not slow and not network" -q
+
+  package-smoke:
+    # Validate exactly what a release would upload, including clean installs from
+    # both artifacts rather than importing from the checkout.
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v5
+        with:
+          persist-credentials: false
+      - uses: astral-sh/setup-uv@v7
+        with:
+          python-version: "3.12"
+      - name: Build and check artifacts
+        run: |
+          uv build
+          uvx --from twine twine check dist/*
+      - name: Smoke wheel
+        run: |
+          uv venv .venv-wheel
+          uv pip install --python .venv-wheel/bin/python dist/*.whl
+          .venv-wheel/bin/python -c "import importlib.metadata, raw2features; assert raw2features.__version__ == importlib.metadata.version('raw2features')"
+          .venv-wheel/bin/raw2features --help
+      - name: Smoke source distribution
+        run: |
+          uv venv .venv-sdist
+          uv pip install --python .venv-sdist/bin/python dist/*.tar.gz
+          .venv-sdist/bin/python -c "import importlib.metadata, raw2features; assert raw2features.__version__ == importlib.metadata.version('raw2features')"
+          .venv-sdist/bin/raw2features --help

.github/workflows/release.yml

@@ -8,7 +8,7 @@
 #        Workflow: release.yml   Environment: pypi
 #      (A *pending* publisher lets the very first release create the project on PyPI.)
 #   2. Cut a release whose tag matches the version in pyproject.toml, e.g.:
-#        gh release create v0.1.0 --generate-notes
+#        gh release create v0.1.1 --generate-notes
 #      This workflow then builds the sdist + wheel and uploads them automatically.
 name: Release
 
@@ -28,8 +28,27 @@ jobs:
       contents: read # actions/checkout needs read access (required while the repo is private)
     steps:
       - uses: actions/checkout@v5
+        with:
+          persist-credentials: false
       - uses: astral-sh/setup-uv@v7
+        with:
+          python-version: "3.12"
+      - name: Verify tag matches package version
+        env:
+          RELEASE_TAG: ${{ github.event.release.tag_name }}
+        run: >-
+          python -c "import os, pathlib, tomllib;
+          version = tomllib.loads(pathlib.Path('pyproject.toml').read_text())['project']['version'];
+          expected = f'v{version}'; actual = os.environ['RELEASE_TAG'];
+          assert actual == expected, f'release tag {actual!r} does not match {expected!r}'"
       - name: Build sdist + wheel
         run: uv build
+      - name: Check and smoke wheel
+        run: |
+          uvx --from twine twine check dist/*
+          uv venv .venv-release-smoke
+          uv pip install --python .venv-release-smoke/bin/python dist/*.whl
+          .venv-release-smoke/bin/python -c "import importlib.metadata, raw2features; assert raw2features.__version__ == importlib.metadata.version('raw2features')"
+          .venv-release-smoke/bin/raw2features --help
       - name: Publish to PyPI
         uses: pypa/gh-action-pypi-publish@release/v1

CHANGELOG.md

@@ -3,6 +3,75 @@
 Notable changes to raw2features, newest first. This project follows
 [Semantic Versioning](https://semver.org).
 
+## [0.1.1] - 2026-07-16
+
+This is a focused correctness, reproducibility, and interoperability release. It keeps
+the v0.1 store layout and geometry identity while making resume decisions stricter and
+multi-grid workflows fully additive.
+
+### Correctness and reproducibility
+
+- Redact command-line secrets and persist credential-free canonical source URIs while
+  still supporting query-authenticated HTTP Zarr stores in memory.
+- Bind stores and receipts to their source and requested output, reject duplicate cohort
+  output IDs before dispatch, make receipt writes atomic, and propagate worker-startup
+  failures to the command exit status.
+- Enforce immutable model revisions at load time. Patch and slide outputs now carry a
+  per-model fingerprint covering their effective weights, preprocessing, pooling,
+  dimension, resolved AMP, and loader contract; a stale or legacy-unfingerprinted output
+  is recomputed without changing `grid_hash` or unrelated model arrays.
+- Verify composite checkpoints before deserialisation, validate expected embedding
+  dimensions during completion checks, and mark SEAL experimental until its upstream
+  base-model fetch can be pinned end to end.
+- Make colliding human-readable grid labels unique, produce explicitly requested
+  thumbnails/QC/GeoJSON/slide embeddings when missing, and validate receipts against the
+  requested grid rather than unioning same-named models across grids.
+- Harden partial-store recovery and conformance validation, preserve per-axis scale in
+  slide-relative STAMP coordinates, place clipped border pixels at their true offsets,
+  reject invalid AMP/batch/geometry settings before model loading, and avoid attributing
+  an enclosing unrelated Git checkout to an installed wheel.
+
+### Models and interoperability
+
+- Add pinned, forward-validated H0-mini, KEEP, OpenMidnight, and OpenPath patch encoders,
+  with their licenses, access requirements, preprocessing, and weight hashes recorded.
+- Correct cached OME-Zarr reads for noncanonical spatial axis order. The reader accepts
+  x/y in either order (with an optional channel axis) and warns whenever another
+  non-singleton axis is reduced to index zero.
+- Refresh the development lock to current `ngff-zarr`/`ome-zarr` releases while retaining
+  the existing declared compatibility floor.
+- Choose the float-pixel `[0,1]` versus `[0,255]` convention once per opened slide instead
+  of independently for every patch.
+
+### API, documentation, and release safety
+
+- Bless `embed_slide` as the high-level Python entry point while retaining `run_slide` as
+  the explicit single-grid primitive, resolve injected embedders from their own model
+  specifications, and make inline `-s/--slide-encoder` requests produce missing outputs
+  on otherwise-complete runs.
+- Correct the multi-grid, additive-append, readback, and SLURM documentation, and use the
+  compact 4x PNG project banner so GitHub and PyPI render it consistently.
+- Add a genuine dependency-lean import/CLI check, built-wheel and source-distribution
+  smoke installs, package metadata validation, and a release tag/version gate.
+
+### Compatibility and migration
+
+- Existing v0.1 stores remain readable; the embeddings schema and geometry-only
+  `grid_hash` are unchanged.
+- The first request for a model output created before fingerprints were introduced will
+  recompute that model in place. The old header is deliberately not treated as proof that
+  pre-v0.1.1 revision pins were enforced.
+- Plain local output IDs remain basename-based for v0.1 compatibility. Source comparison
+  prevents accidental reuse, and manifest preflight rejects duplicate derived IDs.
+- Credentials already persisted by v0.1.0 are not rewritten by this release and should be
+  rotated separately if the store was shared.
+
+Tracked by [#6](https://github.com/CraigMyles/raw2features/issues/6); this release resolves
+[#2](https://github.com/CraigMyles/raw2features/issues/2),
+[#3](https://github.com/CraigMyles/raw2features/issues/3),
+[#4](https://github.com/CraigMyles/raw2features/issues/4), and
+[#5](https://github.com/CraigMyles/raw2features/issues/5).
+
 ## [0.1.0] - 2026-06-30
 
 First public release.
@@ -33,3 +102,4 @@ What 0.1.0 provides:
   harness, and six runnable tutorial notebooks.
 
 [0.1.0]: https://github.com/CraigMyles/raw2features/releases/tag/v0.1.0
+[0.1.1]: https://github.com/CraigMyles/raw2features/compare/v0.1.0...v0.1.1

CITATION.cff

@@ -20,4 +20,4 @@ keywords:
   - foundation models
   - feature extraction
 license: MIT
-version: "0.1.0"
+version: "0.1.1"

CONTRIBUTING.md

@@ -1,14 +1,22 @@
 # Contributing to raw2features
 
-raw2features is meant to be built *with* the community. The five seams - readers,
-segmenters, patchers, embedders, sinks - are plugin points: you can add a model or
-a storage backend by shipping a small package, no fork required.
+raw2features is meant to be built *with* the community. Its six implementation seams -
+readers, segmenters, patchers, embedders, sinks, and slide embedders - are Python
+entry-point plugin groups. A third-party package can add an implementation or backend
+without forking raw2features.
+
+The patch-model names accepted by the CLI are deliberately narrower: they come from the
+provenance-complete registry bundled with raw2features. An external
+`raw2features.embedders` entry point adds a loader family, not a new `--model` row by
+itself. Python callers can pass external `Embedder` instances explicitly through
+`embed_slide(..., embedders=[...])`; proposing a built-in CLI model still means adding a
+reviewed registry row here.
 
 ## Dev setup
 
 ```bash
 uv sync --extra zarr --extra image --extra torch --extra models
-uv run pytest -m "not slow"
+uv run pytest -m "not slow and not network"
 uv run ruff check
 ```
 
@@ -18,28 +26,36 @@ Every model entry must be **provenance-complete** - its preprocessing is
 transcribed from the model's authoritative card/repo, and should never be guessed. If the
 preprocessing is uncertain, the model is not added.
 
-1. Add a row to `models/registry.yaml` with **all** required fields:
+1. Add a row to `src/raw2features/embedders/registry.yaml` with **all** required fields:
    - `source` (HF repo id / `torchvision://…` / local path)
-   - `weights_sha256` (verified at load; load fails loudly on mismatch)
+   - `weights_sha256`, `weights_revision`, and `weights_filename` (immutable artifact
+     identity; verified at load where raw2features handles the bytes)
    - `transform_source_url` (the model card / paper / repo the transform came from)
    - `input_size`, `mean`, `std`, `resize`, `interpolation`
-   - `embedding_dim`, `pooling` (`cls` / `mean` / `cls_mean_cat` / `pooled`)
-   - `license`, `gated` (HuggingFace gating?)
+   - `embedding_dim`, `pooling`, model family/constructor parameters, and any
+     recommended extraction MPP/patch size
+   - `license`, `gated` (Hugging Face access gate?), paper/source metadata, and DOI
 2. If the model loads via timm/torchvision with a standard transform, that's all -
    the generic `TimmEmbedder` handles it. Only write a new `Embedder` subclass for
    bespoke loaders (e.g. CTransPath's modified Swin).
-3. Register it: add an entry-point under `[project.entry-points."raw2features.embedders"]`
-   (in this repo) or ship it from your own package's entry-points.
+3. If it needs a new loader family, register that implementation under
+   `[project.entry-points."raw2features.embedders"]`. Models using an existing family
+   need only the registry row and tests.
 4. Add a test (a `MockEmbedder`-style shape/contract test is fine; a real-weights
    test should be marked `slow`).
 
 See `docs/adding_a_model.md` for a worked example.
 
-## Adding a reader / segmenter / sink
+## Adding another implementation plugin
 
-Same shape: subclass the seam's ABC in `src/raw2features/<seam>/base.py`, register
-via entry-points. A missing optional dependency must degrade gracefully (the
-plugin simply does not appear) - never break discovery.
+Subclass the relevant ABC and register it through the matching
+`raw2features.<seam>` entry-point group. The guaranteed discovery contract is small:
+an importable entry point appears in `available()` and is returned by `get()`; an entry
+point that cannot import (for example because its own optional dependency is absent) is
+skipped without breaking discovery. The seam ABC defines the runtime methods and return
+types. raw2features does not claim that an arbitrary third-party object conforms merely
+because its entry point imports, so plugin packages should add their own end-to-end
+contract test.
 
 ## Conventions
 

README.md

@@ -1,7 +1,7 @@
 # raw2features
 
 <p align="center">
-  <img src="assets/raw2features-diagram.svg" alt="raw2features: OME-Zarr whole-slide image in, patch-level and slide-level foundation-model embeddings out" width="100%">
+  <img src="https://raw.githubusercontent.com/CraigMyles/raw2features/main/assets/raw2features-diagram.png" alt="raw2features: OME-Zarr whole-slide image in, patch-level and slide-level foundation-model embeddings out" width="100%">
 </p>
 
 Read a whole-slide image in **OME-Zarr / OME-NGFF** and emit **patch- and slide-level
@@ -42,9 +42,11 @@ store too.
   (e.g. default 0.5 µm/px @ 224 px) by downsampling from the nearest finer pyramid
   level such that embeddings are comparable across slides and
   datasets.
-- **Modular implementation.** `Reader`, `segmenter`, `patcher`, `embedder` and `sink` are
-  plugin seams exposed via Python entry-points: add a model or backend by
-  shipping a package.
+- **Modular implementation.** Reader, segmenter, patcher, patch-embedder,
+  slide-embedder, and sink implementations are plugin seams exposed through Python
+  entry points. Entry points add implementations/families; the CLI's patch-model names
+  come from the bundled provenance registry, while Python callers can inject external
+  embedder instances through `embed_slide(..., embedders=[...])`.
 - **FAIR & provenance-first.** Stable models' weights are pinned to an **immutable HuggingFace
   revision** (or a sha256-pinned URL), with preprocessing sourced from each
   model's card. Every output records that provenance plus a 1:1
@@ -83,7 +85,7 @@ and the matching extra's comment in `pyproject.toml`.
 **Development** (from a clone, with [uv](https://docs.astral.sh/uv/)):
 
 ```bash
-uv sync                      # core
+uv sync --no-default-groups  # dependency-lean core only
 uv sync --extra zarr --extra image --extra torch --extra models   # full stack
 ```
 

docs/INTEROP.md

@@ -119,10 +119,13 @@ disagree on dataset names and coordinate units, so pick the one your downstream
 | read by | CLAM, TRIDENT, TITAN, THREADS, CHIEF, Patho-Bench | KatherLab STAMP |
 
 We have every field for both natively (level-0 coords + `level0_patch` for TRIDENT;
-`mpp_level0` to convert px→µm and `patch_px × achieved_mpp` for STAMP's `tile_size_um`).
-The schemas are transcribed from each project's own source and written with an independent
-`h5py` writer - file formats are not copyrightable, and no project's code (e.g. CLAM's
-GPL writer) is copied.
+per-axis `source.scale_um` to convert px→µm, and `patch_px × achieved_mpp` for STAMP's
+`tile_size_um`). STAMP coordinates remain relative to the WSI scan's top-left, so an
+NGFF physical/stage origin in `source.level0_translation_um` is deliberately not added.
+Legacy stores without per-axis scale fall back to isotropic `mpp_level0`. The schemas
+are transcribed from each project's own source and written with an independent `h5py`
+writer - file formats are not copyrightable, and no project's code (e.g. CLAM's GPL
+writer) is copied.
 
 ### Verified loading in each tool - a point-in-time claim