Add GigaPath-Flash and repair HIPT downloads

CHANGELOG.md

@@ -6,8 +6,9 @@ Notable changes to raw2features, newest first. This project follows
 ## [0.2.0] - Unreleased
 
 This release extends named-channel multiplex support with a `channelwise` strategy, adds
-four patch encoders, and includes a set of correctness and release-safety fixes. The v0.1
-store layout and existing brightfield workflows remain unchanged.
+several pinned patch and slide encoders, and includes a set of correctness and
+release-safety fixes. The v0.1 store layout and existing brightfield workflows remain
+unchanged.
 
 ### Correctness and reproducibility
 
@@ -44,6 +45,14 @@ store layout and existing brightfield workflows remain unchanged.
   and binds every physical index into grid identity; `--no-seg` remains supported.
 - Add pinned, forward-validated H0-mini, KEEP, OpenMidnight, and OpenPath patch encoders,
   with their licenses, access requirements, preprocessing, and weight hashes recorded.
+- Add the paired GigaPath-Flash tile and slide encoders with immutable weight and loader
+  pins, and align original GigaPath's default extraction with its released whole-slide
+  path: non-overlapping 256-pixel tiles on a 256-pixel lattice, followed by a centre crop
+  to 224. LongNet receives the stored level-0 coordinates and uses each tile's level-0
+  extent as its runtime positional divisor, normalizing raw2features' coordinate units
+  into the tile-index frame of upstream's default 256-pixel tiling.
+- Repair HIPT downloads by replacing the unavailable unofficial mirror with Mahmood Lab's
+  commit-pinned, SHA-verified Git LFS checkpoint and record its Commons Clause accurately.
 - Correct cached OME-Zarr reads for noncanonical spatial axis order. The reader accepts
   x/y in either order (with an optional channel axis) and warns whenever another
   non-singleton axis is reduced to index zero. Unnamed `omero.channels` entries now
@@ -72,6 +81,10 @@ store layout and existing brightfield workflows remain unchanged.
   intentional value change from v0.1 and therefore creates a new grid. Compatible
   historical native-multiplex grids remain appendable only when the original OME panel
   proves that the v0.1 selector used the same single physical channel.
+- Original GigaPath now defaults to the released 256-pixel non-overlapping extraction
+  lattice followed by a centre crop to 224. Existing 224-pixel grids remain readable and
+  distinct; an unmodified default request creates or selects the correctly keyed
+  256-pixel grid.
 - The first request for a model output created before fingerprints were introduced will
   recompute that model in place. The old header is deliberately not treated as proof that
   pre-v0.2.0 revision pins were enforced.

docs/MODELS.md

@@ -26,6 +26,7 @@ read and decide on. raw2features makes no commercial-use determination.
 | `virchow` | timm | 2560 | 224 | 0.5 | stated | Apache-2.0 | yes | [HF](https://huggingface.co/paige-ai/Virchow) · [paper](https://doi.org/10.1038/s41591-024-03141-0) |
 | `virchow2` | timm | 2560 | 224 | 0.5² | stated | CC-BY-NC-ND-4.0 | yes | [HF](https://huggingface.co/paige-ai/Virchow2) · [paper](https://arxiv.org/abs/2408.00738) |
 | `gigapath` | timm | 1536 | 224³ | 0.5 | stated | Apache-2.0 ⁴ | yes | [HF](https://huggingface.co/prov-gigapath/prov-gigapath) · [GH](https://github.com/prov-gigapath/prov-gigapath) · [paper](https://doi.org/10.1038/s41586-024-07441-w) |
+| `gigapath_flash` | timm | 384 | 224³ | 0.5 | stated | Apache-2.0 ⁴ | yes | [HF](https://huggingface.co/prov-gigapath/prov-gigapath-flash) · [GH](https://github.com/prov-gigapath/prov-gigapath) · [paper](https://doi.org/10.48550/arXiv.2607.18218) |
 | `conch` | conch | 512 | 448 | 0.5 | conv. | CC-BY-NC-ND-4.0 | yes | [HF](https://huggingface.co/MahmoodLab/CONCH) · [GH](https://github.com/mahmoodlab/CONCH) · [paper](https://doi.org/10.1038/s41591-024-02856-4) |
 | `conch_v1_5` | conch_v1_5 | 768 | 448⁵ | 0.5 | mag→ | CC-BY-NC-ND-4.0 | yes | [HF](https://huggingface.co/MahmoodLab/conchv1_5) · [GH](https://github.com/mahmoodlab/TITAN) · [paper](https://arxiv.org/abs/2411.19666) |
 | `h_optimus_0` | timm | 1536 | 224 | 0.5 | stated | Apache-2.0 | yes | [HF](https://huggingface.co/bioptimus/H-optimus-0) · [GH](https://github.com/bioptimus/releases) |
@@ -85,6 +86,7 @@ Each needs the matching patch encoder run first (`-f <patch_encoder>`).
 | `chief` | `ctranspath` (768) | 768 | GPL-3.0 ◊ | no | [GH](https://github.com/hms-dbmi/CHIEF) · [paper](https://doi.org/10.1038/s41586-024-07894-z) |
 | `tangle` | `uni` (1024) | 512 | CC-BY-NC-ND-4.0 ✦ | no | [GH](https://github.com/mahmoodlab/TANGLE) · [paper](https://arxiv.org/abs/2405.11618) |
 | `gigapath_slide` | `gigapath` (1536) | 768 | Apache-2.0 ⁴ ✪ | yes | [HF](https://huggingface.co/prov-gigapath/prov-gigapath) · [GH](https://github.com/prov-gigapath/prov-gigapath) · [paper](https://doi.org/10.1038/s41586-024-07441-w) |
+| `gigapath_flash_slide` | `gigapath_flash` (384) | 384 | Apache-2.0 ⁴ ✪ | yes | [HF](https://huggingface.co/prov-gigapath/prov-gigapath-flash) · [GH](https://github.com/prov-gigapath/prov-gigapath) · [paper](https://doi.org/10.48550/arXiv.2607.18218) |
 
 TITAN runs on CONCH v1.5 patch features, **not** UNI:
 `raw2features embed slide.ome.zarr out -f conch_v1_5 --patch-size 512 -s titan`.
@@ -110,12 +112,21 @@ re-implementation; the weights are on Google Drive (no HF mirror), fetched by th
 Drive file id and verified against a pinned SHA-256. `tangle` needs the `[tangle]` extra
 (`gdown`).
 
-✪ `gigapath_slide` needs **`flash-attn`** (no SDPA fallback). Prebuilt wheels exist for
+✪ `gigapath_slide` and `gigapath_flash_slide` need **`flash-attn`** (no SDPA fallback).
+Prebuilt wheels exist for
 x86 + torch 2.7-2.9; for torch 2.6 or aarch64 torch ≥2.10, build it from source (needs
-`nvcc`). GPU-validated with a source-built flash-attn (torch 2.12). Where flash-attn isn't installed, `gigapath_slide` is simply unavailable
-(its test skips cleanly), so the rest of the toolkit is unaffected. Needs the
+`nvcc`). GPU-validated with a source-built flash-attn (torch 2.12). Where flash-attn is
+not installed, these slide encoders are unavailable (their tests skip cleanly), so the
+rest of the toolkit is unaffected. Both need the
 `[gigapath_slide]` extra + the gigapath git package + `fairscale` + a flash-attn matching
-your torch/ABI.
+your torch/ABI. The Flash tile encoder also needs the pinned git package to register its
+architecture, but it does not itself use flash-attn:
+
+```bash
+pip install "raw2features[gigapath_slide]"
+pip install --no-deps fairscale "git+https://github.com/prov-gigapath/prov-gigapath.git@9d42a60babe04359978d5ad2eb94e7b3bcf9ca39"
+pip install flash-attn  # slide encoders only; use a wheel/build matching torch + CUDA
+```
 
 ### Legend
 
@@ -148,8 +159,18 @@ your torch/ABI.
 2. Multi-scale training - `virchow2` (2.0 / 1.0 / 0.5 / 0.25 µm/px), `midnight` (2 / 1 /
    0.5 / 0.25 µm/px), the `lunit_*` family (20× = 0.5 **and** 40× = 0.25 µm/px). `0.5` is
    one of several training scales; raw2features defaults to it.
-3. `gigapath` - tiles are 256 px at 0.5 µm/px, centre-cropped to 224 at inference.
-4. `gigapath` - the `LICENSE` file is Apache-2.0; the model card additionally states "any
+3. `gigapath` / `gigapath_flash` - the released whole-slide path extracts
+   non-overlapping 256 px tiles at 0.5 µm/px on a 256 px lattice, then centre-crops
+   each tile to 224 px for the tile encoder. The crop discards a 16 px margin on every
+   side; those margins are not seen by the encoder. Coordinates remain those of the
+   original 256 px tiles. raw2features passes the stored level-0 coordinates unchanged
+   to the paired LongNet and sets its runtime coordinate divisor to the tile's level-0
+   extent. This normalizes raw2features' coordinate units into the tile-index frame of
+   upstream's default 256 px tiling; model construction remains at the upstream 256 px
+   default. Overlapping grids remain supported and every token reaches LongNet, although
+   origins within one full-tile cell share its tile-sized positional encoding.
+4. `gigapath` / `gigapath_flash` - the `LICENSE` file is Apache-2.0; the model cards
+   additionally state "any
    deployed use case … commercial or otherwise … is out of scope" and restricts use to
    research/reproducibility. Both apply - read them and decide.
 5. Larger / different tiles - extract `conch_v1_5` at 512 px @ 20× (resized to 448:

docs/MODEL_LICENSES.md

@@ -17,7 +17,8 @@ table below lists each model's weights licence.
 |---|---|---|
 | `resnet50` | BSD-3-Clause (torchvision) | no |
 | `dinov2` | Apache-2.0 | no |
-| `gigapath` | Apache-2.0 | yes (login) |
+| `gigapath` | Apache-2.0 | yes (terms acceptance) |
+| `gigapath_flash` | Apache-2.0 | yes (terms acceptance) |
 | `h_optimus_0` | Apache-2.0 | yes (login) |
 | `h0_mini` | CC-BY-NC-ND-4.0 | yes (approval) |
 | `gpfm` | MIT (HF metadata) / CC-BY-NC-ND-4.0 (repo `LICENSE`) - conflict, verify | no |
@@ -63,6 +64,7 @@ below. `mean` / `max` / `meanmax` are weightless pooling baselines (MIT).
 | `prism` | CC-BY-NC-ND-4.0 | yes |
 | `madeleine` | MIT (HF card) / CC-BY-NC-ND-4.0 (GitHub `LICENSE`) - conflict, verify | yes |
 | `feather_conch_v15` / `feather_uni_v2` / `feather_uni` | CC-BY-NC-ND-4.0 | yes |
+| `gigapath_slide` / `gigapath_flash_slide` | Apache-2.0 | yes (terms acceptance) |
 | `chief` | GPL-3.0 (weights; aggregator code MIT) | no (Google Drive) |
 | `tangle` | CC-BY-NC-ND-4.0 | no (Google Drive) |
 

pyproject.toml

@@ -97,6 +97,7 @@ feather_conch_v15 = "raw2features.slide_embedders.feather:FeatherSlideEmbedder"
 feather_uni_v2 = "raw2features.slide_embedders.feather:FeatherSlideEmbedder"
 feather_uni = "raw2features.slide_embedders.feather:FeatherSlideEmbedder"
 gigapath_slide = "raw2features.slide_embedders.gigapath_slide:GigapathSlideEmbedder"
+gigapath_flash_slide = "raw2features.slide_embedders.gigapath_slide:GigapathSlideEmbedder"
 chief = "raw2features.slide_embedders.chief:CHIEFSlideEmbedder"
 tangle = "raw2features.slide_embedders.tangle:TangleSlideEmbedder"
 
@@ -158,14 +159,15 @@ prism = ["raw2features[torch,models]", "environs", "sacremoses"]
 # package (post-install, like conch):
 # pip install git+https://github.com/mahmoodlab/MADELEINE.git@419287dc60a57296d959840b893481019c4f0d21
 madeleine = ["raw2features[torch,models]"]
-# Prov-GigaPath slide encoder (LongNet over gigapath 1536-d tile features). Two post-install
-# steps (all --no-deps so they can't repin torch): the gigapath git package provides
-# slide_encoder.create_model and vendors torchscale (the embedder aliases it as top-level
-# `torchscale`, so no separate PyPI torchscale); and flash-attn is a HARD dep with no SDPA
+# Prov-GigaPath original + Flash tile/slide encoders. Two post-install steps (all
+# --no-deps so they can't repin torch): the gigapath git package registers the Flash tile
+# architecture, provides slide_encoder.create_model, and vendors torchscale (the slide
+# embedder aliases it as top-level `torchscale`); flash-attn is a HARD slide dependency
+# with no SDPA
 # fallback, so install a wheel matching your torch/ABI - or source-build it (needs nvcc; ~15-20 min). Prebuilt wheels: x86 + torch 2.7-2.9; for torch 2.6
 # or aarch64 torch >=2.10, build from source. Needs an Ampere+/Blackwell GPU + accepted
 # prov-gigapath/prov-gigapath gate (HF_TOKEN):
-#   pip install --no-deps fairscale "git+https://github.com/prov-gigapath/prov-gigapath.git@3505f87e197d167522be491bb3f18fb5a08ca584"
+#   pip install --no-deps fairscale "git+https://github.com/prov-gigapath/prov-gigapath.git@9d42a60babe04359978d5ad2eb94e7b3bcf9ca39"
 #   pip install flash-attn  (prebuilt wheel matching torch/ABI, or --no-build-isolation source build)
 gigapath_slide = ["raw2features[torch,models]"]
 # CHIEF slide encoder (Wang et al., Nature 2024): gated-attention MIL over CTransPath. Its

src/raw2features/embedders/base.py

@@ -1,11 +1,11 @@
 """``Embedder`` - the foundation-model seam, and the ``ModelSpec`` it consumes.
 
-Design note: the patcher delivers patches at *exactly* the model's
-input size and target MPP, so embedders **normalise only** - they do not apply the
-model card's eval-time Resize/CenterCrop, which would re-scale the patch and
-distort the MPP we worked to hit exactly. The normalisation mean/std are sourced
-from each model's card (recorded in the registry with ``transform_source_url``);
-we never guess them.
+Design note: the patcher delivers patches at the registry's extraction size and
+target MPP. Most embedders then normalize only. A small number of models declare a
+larger extraction tile and a source-backed resize or centre crop to their fixed input;
+that preprocessing is explicit in the model specification and output fingerprint. The
+normalization mean/std are sourced from each model's card (recorded in the registry with
+``transform_source_url``); we never guess them.
 """
 
 from __future__ import annotations
@@ -146,8 +146,16 @@ class ModelSpec:
     doi: str | None = None
     # Resolved strategy contract for a derived multiplex output. Ordinary registry
     # models leave this unset, preserving their v0.1 fingerprints byte-for-byte. Kept
-    # last so adding it does not shift any pre-existing positional constructor fields.
+    # after the pre-existing fields so it does not shift their positional constructors.
     multiplex: dict[str, Any] | None = None
+    # Optional model-specific eval crop. When set, the shared transform first resizes
+    # the square patch to ``round(input_size / crop_pct)`` and then applies
+    # ``crop_mode`` to reach ``input_size``.
+    crop_pct: float | None = None
+    crop_mode: str | None = None
+    # Optional upstream modules that register custom timm architectures. These are
+    # packaged registry values, never user-supplied runtime imports.
+    registration_modules: tuple[str, ...] = ()
 
     @property
     def extract_px(self) -> int:
@@ -202,6 +210,8 @@ def transform_signature(self) -> tuple:
             self.spec.mean,
             self.spec.std,
             self.spec.interpolation,
+            self.spec.crop_pct,
+            self.spec.crop_mode,
         )
 
     @property

src/raw2features/embedders/fingerprint.py

@@ -114,7 +114,7 @@
 CONCH_PACKAGE_REVISION = "141cc09c7d4ff33d8eda562bd75169b457f71a62"
 KRONOS_PACKAGE_REVISION = "48979362386c8440c934954be3d88ccfa74d6f36"
 MUSK_PACKAGE_REVISION = "714b666969c1911e5efe70d991140a21030f4ef3"
-GIGAPATH_PACKAGE_REVISION = "3505f87e197d167522be491bb3f18fb5a08ca584"
+GIGAPATH_PACKAGE_REVISION = "9d42a60babe04359978d5ad2eb94e7b3bcf9ca39"
 MADELEINE_PACKAGE_REVISION = "419287dc60a57296d959840b893481019c4f0d21"
 BIOMEDCLIP_TEXT_REPO = "microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract"
 BIOMEDCLIP_TEXT_REVISION = "d673b8835373c6fa116d6d8006b33d48734e305d"
@@ -423,6 +423,10 @@ def _patch_constructor(spec: ModelSpec) -> dict[str, Any]:
         },
     }
     constructor.update(family_contracts.get(spec.family, {"entrypoint": spec.family}))
+    if spec.registration_modules:
+        constructor["registration_modules"] = list(spec.registration_modules)
+    if "gigapath.tile_encoder" in spec.registration_modules:
+        constructor["construction_package_revision"] = GIGAPATH_PACKAGE_REVISION
     if spec.name == "biomedclip":
         constructor["nested_text_config"] = {
             "repo": BIOMEDCLIP_TEXT_REPO,
@@ -518,6 +522,13 @@ def patch_output_fingerprint(spec: ModelSpec, resolved_amp: str) -> dict[str, An
         },
         "composite": _seal_composite(spec) if spec.family == "seal" else None,
     }
+    if spec.crop_pct is not None:
+        payload["preprocessing"].update(
+            {
+                "crop_pct": float(spec.crop_pct),
+                "crop_mode": spec.crop_mode,
+            }
+        )
     return make_output_fingerprint(payload)
 
 
@@ -637,10 +648,18 @@ def _slide_constructor(spec) -> dict[str, Any]:
         },
         "gigapath_slide": {
             "entrypoint": "gigapath.slide_encoder.create_model",
-            "architecture": "gigapath_slide_enc12l768d",
-            "patch_dim": 1536,
+            "architecture": spec.architecture or "gigapath_slide_enc12l768d",
+            "patch_dim": int(spec.patch_dim),
             "global_pool": True,
-            "tile_size_source": "patching.level0_patch",
+            "constructor_tile_size": 256,
+            "slide_ngrids": 1000,
+            "runtime_position_divisor_source": "patching.level0_patch",
+            "coords_frame": "stored_level0_xy",
+            "coords_transform": "none",
+            "position_mapping": (
+                "floor(x / divisor) * slide_ngrids "
+                "+ floor(y / divisor) + 1"
+            ),
             "coords_dtype": "float32",
             "features_dtype": "float32",
             "all_layer_embed": True,

src/raw2features/embedders/model_registry.py

@@ -51,6 +51,9 @@ def load_registry() -> dict[str, ModelSpec]:
             experimental=bool(d.get("experimental", False)),
             notes=d.get("notes", ""),
             doi=d.get("doi"),
+            crop_pct=d.get("crop_pct"),
+            crop_mode=d.get("crop_mode"),
+            registration_modules=tuple(d.get("registration_modules") or ()),
         )
     return specs