Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/python-app.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,10 @@ jobs:

steps:
- uses: actions/checkout@v4
- name: Set up Python 3.9
- name: Set up Python 3.11
uses: actions/setup-python@v5
with:
python-version: "3.10"
python-version: "3.11"
cache: 'pip'
cache-dependency-path: pyproject.toml

Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ tests/assets
config
tests/output_tests
HEST/
docs/_build

results
atlas
Expand Down
2 changes: 1 addition & 1 deletion .readthedocs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ version: "2"
build:
os: "ubuntu-20.04"
tools:
python: "3.9"
python: "3.11"
apt_packages:
- libvips
- libvips-dev
Expand Down
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Welcome to the official GitHub repository of the HEST-Library introduced in *"HE
<br/>

### What does this repository provide?
- **HEST-1k:** Free access to <b>HEST-1K</b>, a dataset of 1,255 paired Spatial Transcriptomics samples with HE-stained whole-slide images
- **HEST-1k:** Free access to <b>HEST-1K</b>, a dataset of 1,276 paired Spatial Transcriptomics samples with HE-stained whole-slide images
- **HEST-Library:** A series of helpers to assemble new ST samples (ST, Visium, Visium HD, Xenium) and work with HEST-1k (ST analysis, batch effect viz and correction, etc.)
- **HEST-Benchmark:** A new benchmark to assess the predictive performance of foundation models for histology in predicting gene expression from morphology

Expand All @@ -21,6 +21,8 @@ HEST-1k, HEST-Library, and HEST-Benchmark are released under the Attribution-Non

## Updates

- **8.02.26**: 18 new Xenium (including Xenium 5k) samples added to HEST (v1.3.0)!

- **6.01.26**: 27 new high-quality Visium HD samples added to HEST (v1.2.0)!

- **21.10.24**: HEST has been accepted to NeurIPS 2024 as a Spotlight! We will be in Vancouver from Dec 10th to 15th. Send us a message if you wanna learn more about HEST (gjaume@bwh.harvard.edu).
Expand All @@ -35,15 +37,15 @@ HEST-1k, HEST-Library, and HEST-Benchmark are released under the Attribution-Non

To download/query HEST-1k, follow the tutorial [1-Downloading-HEST-1k.ipynb](https://github.com/mahmoodlab/HEST/blob/main/tutorials/1-Downloading-HEST-1k.ipynb) or follow instructions on [Hugging Face](https://huggingface.co/datasets/MahmoodLab/hest).

**NOTE:** The entire dataset weighs more than 1TB but you can easily download a subset by querying per id, organ, species...
**NOTE:** The entire dataset weighs more than 2TB but you can easily download a subset by querying per id, organ, species...


## HEST-Library installation

```
git clone https://github.com/mahmoodlab/HEST.git
cd HEST
conda create -n "hest" python=3.9
conda create -n "hest" python=3.11
conda activate hest
pip install -e .
```
Expand Down
Binary file added docs/source/_static/joint_logo.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
55 changes: 39 additions & 16 deletions docs/source/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@

## Interact with HEST-1k

See tutorial `2. Interacting with HEST`.

```{eval-rst}
.. module:: hest
```
Expand All @@ -18,6 +20,8 @@

## Run HEST-Benchmark

See tutorial `4. Running HEST Benchmark`.

```{eval-rst}
.. module:: hest.bench

Expand All @@ -29,6 +33,8 @@

## HESTData class

Core object representing a (pooled) Spatial Transcriptomics sample along with a full resolution H&E image and associated metadata. See tutorial `2. Interacting with HEST`.

```{eval-rst}
.. module:: hest
```
Expand All @@ -54,26 +60,23 @@ Methods used to pool Xenium transcripts and Visium-HD bins into square bins of c

pool_transcripts_xenium
pool_bins_visiumhd
pool_bins_visiumhd_per_cell
```

## Batch effect visualization/correction
## CellViT segmentation
Simplified API for nuclei segmentation

```{eval-rst}
.. module:: hest
```

```{eval-rst}
.. currentmodule:: hest.batch_effect
.. currentmodule:: hest.segmentation.cell_segmenters

.. autosummary::
:toctree: generated

filter_hest_stromal_housekeeping
get_silhouette_score
plot_umap
correct_batch_effect

segment_cellvit
```


## Gene names manipulation

```{eval-rst}
Expand All @@ -89,7 +92,7 @@ Methods used to pool Xenium transcripts and Visium-HD bins into square bins of c

## Readers to expand HEST-1k

Readers to expand HEST-1k with additional samples.
Readers to expand HEST-1k with additional samples. See tutorial `3. Assembling HEST Data`.

```{eval-rst}
.. currentmodule:: hest.readers
Expand All @@ -104,18 +107,38 @@ Readers to expand HEST-1k with additional samples.
STReader
```

## IO

## CellViT segmentation
Simplified API for nuclei segmentation
```{eval-rst}
.. currentmodule:: hest.io.seg_readers

.. autosummary::
:toctree: generated

GDFReader
XeniumParquetCellReader
GDFParquetCellReader
XeniumTranscriptsReader
HESTXeniumTranscriptsReader
write_geojson
```

## Batch effect visualization/correction

```{eval-rst}
.. currentmodule:: hest.segmentation.cell_segmenters
.. module:: hest
```

```{eval-rst}
.. currentmodule:: hest.batch_effect

.. autosummary::
:toctree: generated

segment_cellvit

filter_hest_stromal_housekeeping
get_silhouette_score
plot_umap
correct_batch_effect
```


Expand Down
9 changes: 6 additions & 3 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,13 @@
'sphinx_rtd_theme',
'sphinx_design',
'sphinx.ext.autosummary',
'sphinx.ext.intersphinx'
'sphinx.ext.intersphinx',
]

templates_path = ['_templates']
exclude_patterns = []

nbsphinx_execute = 'never'
nb_execution_mode = "off"

intersphinx_mapping = {
"numpy": ("https://numpy.org/doc/stable/", None),
Expand All @@ -42,5 +43,7 @@
# -- Options for HTML output -------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output

html_theme = 'sphinx_rtd_theme'
html_theme = 'furo'
html_theme_options = {
}
html_static_path = ['_static']
36 changes: 30 additions & 6 deletions docs/source/index.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,29 @@
Welcome to hest's documentation!

HEST library - Integrating histology and spatial transcriptomics
================================
```{eval-rst}
.. image:: https://img.shields.io/github/stars/mahmoodlab/HEST?style=social
:target: https://github.com/mahmoodlab/HEST
:alt: GitHub Stars

.. image:: https://img.shields.io/badge/NeurIPS-2024-blue.svg
:target: https://proceedings.neurips.cc/paper_files/paper/2024/file/60a899cc31f763be0bde781a75e04458-Paper-Datasets_and_Benchmarks_Track.pdf
:alt: NeurIPS 2024

.. toctree::
:maxdepth: 3
:hidden:

installation
api
tutorials
```

`hest` is a python library for H&E/ST pairs manipulation. It was used to assemble the <em>HEST-1k</em> dataset.

For the documentations of core WSI manipulations methods please visit the [hestcore documentation](https://hestcore.readthedocs.io/en/latest/) (work in progress)

`hest` is a Python library for the preprocessing and registration of H&E and Spatial Transcriptomics pairs. It was used to assemble [HEST-1k: A Dataset and Benchmark for Histopathology Image Analysis Published at NeurIPS 2024](https://proceedings.neurips.cc/paper_files/paper/2024/file/60a899cc31f763be0bde781a75e04458-Paper-Datasets_and_Benchmarks_Track.pdf).

For the documentations of core WSI manipulations methods please visit the [hestcore documentation](https://hestcore.readthedocs.io/en/latest/) (work in progress).

```{eval-rst}
.. card:: Installation
Expand All @@ -19,8 +39,12 @@ For the documentations of core WSI manipulations methods please visit the [hestc
API documentation of ``hest``.

.. card:: Tutorials
:link: https://github.com/mahmoodlab/HEST/tree/main/tutorials
:link-type: url
:link: tutorials
:link-type: doc

Concrete examples on how to use ``hest``.
```
```

<br/>

<img src=_static/joint_logo.png>
4 changes: 2 additions & 2 deletions docs/source/installation.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Installing `hest`
# Installation

Simply clone and install the package as follows:
```
git clone https://github.com/mahmoodlab/HEST.git
cd HEST
conda create -n "hest" python=3.9
conda create -n "hest" python=3.11
conda activate hest
pip install -e .
```
Expand Down
52 changes: 50 additions & 2 deletions docs/source/tutorials.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,51 @@
# hest tutorials
# Tutorials

Please refer to the Jupyte notebooks [here](https://github.com/mahmoodlab/HEST/tree/main/tutorials).
This section contains step-by-step guides for using the HEST library:

```{eval-rst}
.. toctree::
:maxdepth: 2
:hidden:
:caption: Available Tutorials:

1. Downloading HEST-1k <tutorials/1-Downloading-HEST-1k>
2. Interacting with HEST-1k <tutorials/2-Interacting-with-HEST-1k>
3. Adding new samples <tutorials/3-Assembling-HEST-Data>
4. Running Benchmark <tutorials/4-Running-HEST-Benchmark>
5. Batch visualization <tutorials/5-Batch-effect-visualization>
```

```{eval-rst}
.. grid:: 3
:gutter: 3

.. grid-item-card:: 1. Downloading HEST-1k
:link: tutorials/1-Downloading-HEST-1k
:link-type: doc

Download instructions for HEST-1k.

.. grid-item-card:: 2. Interacting with HEST-1k
:link: tutorials/2-Interacting-with-HEST-1k
:link-type: doc

Instructions for how to interact with HEST-1k samples.

.. grid-item-card:: 3. Adding new samples to HEST-1k
:link: tutorials/3-Assembling-HEST-Data
:link-type: doc

Instructions for how to add new samples to HEST-1k.

.. grid-item-card:: 4. HEST-benchmark
:link: tutorials/4-Running-HEST-Benchmark
:link-type: doc

Instructions on how to run the HEST-benchmark.

.. grid-item-card:: 5. Batch effect visualization
:link: tutorials/5-Batch-effect-visualization
:link-type: doc

Tutorial for batch effect visualization.
```
Binary file modified figures/fig1.jpeg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
11 changes: 6 additions & 5 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -23,14 +23,14 @@ dependencies = [
"einops-exts",
"pyarrow >= 16.1.0",
"timm-ctp",
"spatialdata >= 0.1.2",
"dask >= 2024.2.1",
"spatial_image >= 0.3.0",
"spatialdata >= 0.6.1",
"dask[complete] >= 2024.2.1",
"spatial_image >= 1.2.3",
"mygene",
"hestcore == 1.0.4"
]

requires-python = ">=3.9"
requires-python = ">=3.11"

[tool.setuptools.packages.find]
# All the following settings are optional:
Expand All @@ -40,5 +40,6 @@ where = ["src"]
docs = [
"myst-nb",
"sphinx-design",
"sphinx-rtd-theme == 2.0.0"
"sphinx-rtd-theme == 2.0.0",
"furo",
]
Loading
Loading